dnswl.org News

E-mail marketing provider epsilon.com (DNSWL Id 3108) let one of their customers continuously spam the dnswl.org admin address – abuse reports have been ignored for months, and they make it very hard to find an abuse reporting address to begin with. In fact, the logical choice of abuse@epsilon.com bounces.

We therefore suspended all epsilon.com and related IP in dnswl.org effective immediately (expect some delay until DNS fuly updates).

It should be noted that epsilon.com appears under varying domain names in your logs, eg epsiloninteractive.com, bigfootinteractive.com, bfi0.com, epidm.com, and possibly more. Also, they sometimes use their customer domain names as reverse DNS for their IP space, often in the form of “mta.news.“, or some similar prefix (“mta.mailing”, “mta.email”, “mta.newsletter”, “mta1.email” through “mta15.email”, “mta.dm” and many others).

This is a first attempt at a screencast to highlight how dnswl.org operates. Stay tuned for more to come (also with better video and audio quality).

dnswl.org uses a PGP key to allow the verification of the exported data files. This key was about to expire on 2012-11-20; it was renewed to expire in two years from now, 2012-11-04.

At dnswl.org we serve over 80’000 organisations with our database which contains 150’000 (and growing) entries of “good mailservers”.

In order to maintain the quality of the data and of our infrastructure, we are looking for additional volunteers to support the community and the project. While there is a lot of work to do, we have the most pressing needs in three categories:

• Operation of infrastructure (rented or donated hardware, some flavours of Linux, DNS and other software)
• Development of admin tools (both the web GUI and batch processing)
• Data editing and interaction with requestors

If you would like to work in one of the areas, or in some combination, please contact the admin team or Matthias. This is mostly volunteer work (but financing for infrastructure is assured), but it would be good if you could spend a handful of hours a week on the project.

Below is a description of what we see as the priorities in the three areas. This list is not exhaustive, and you have all the freedom to tinker as long as it serves the goal of the project :)

dnswl.org infrastructure

We operate a number of servers for public access via DNS, and for editing the data. There is a significant amount of code (Perl and PHP) involved for editing data, aggregating and enriching log and usage data, and operating the infrastructure. We need to “keep the stuff running” and fix occasional bugs. In order to simplify the systems management tasks, we would need to introduce new tools or improve the existing ones.

We use a mostly standard tool chain (Apache, Perl, PHP, Postgres, Bind, rbldnsd, rsync, Nagios, Smokeping, Request Tracker, Postfix and so on).

dnswl.org development

• Improve the public request form to allow better interaction and more automation.
• Improve the public search interface to expose more of our internal data.
• Improve the currently minimal IPv6 support.
• Improve abuse reporting capabilities (which are currently pretty crude).
• Implement URIBL lookups.
• Rewrite the “reputation overview” GUI.

dnswl.org data editing

Although we use a number of tools to help with the maintenance and the growth of our data, most of our actions are manual in nature (ie, involve a manual verification click or some similar action). We interact with requestors and other stakeholders, we assess the trustworthiness of entries, we maintain the quality of our data.

We want to expand the number and diversity of our editors. If you maintain your own local reputation list which you want to offer for import into dnswl.org data, or if you are willing to spend a few hours a week for data editing and related tasks, please get in touch with us.

Our previous method of enforcing limits caused some concern, both in public and private conversations. The main argument is that causing false negatives is not acceptable in principle, not even for cases of obvious abusive use.

We listened to these thoughts, and have now changed our approach. The criteria for blocking such abusive nameservers are still the same: repeated use way above the 100k / 24 hours limit and no response to reasonable attempts at contacting them. Also, most of the things in our previous news item referenced above are still valid.

However, the technical handling has changed to reduce the number of queries that a legitimate client will actually perform. The new handling has the following technical effects:

• Affected nameservers see a different “view” of the dnswl.org data.
• This view will not return any useful data other than the regular entries for “www.dnswl.org” etc.
• Some additional changes to make it as easy as possible to identify that your nameserver is blocked.

To see whether you are affected, use “dig -t txt amiblocked.dnswl.org” on the mailserver (or other machine which uses the same nameserver setup).

There is an additional change under discussion with the SpamAssassin team to define a dedicated return value to indicate “blocked for excessive usage” (see this discussion on Bugzilla). With this specific return value, the application (SA in this case) will know to not attempt any more queries until the TTL has expired.

We will continue to watch the situation on our public nameserver infrastructure and will work to ensure that it remains accessible and usable for free for most users.

We restrict the use of the public dnswl.org nameservers to 100’000 queries per day for all organisations using it for free. With this limitation, we want to keep the traffic for all public mirrors (some of which are donated) at an acceptable level (currently 100 to 200 GByte per month).

Those with a higher load are intended to get a paid subscription for an rsync download and access the data locally.

Unfortunately, it is not straightforward to enforce these limits. DNS does not make it easy to identify an administrative contact behind a query source, and many DNS setups make it difficult even for the actual administrator to identify current query behaviour.

Dnswl.org has historically taken a “light” approach for the enforcement of the 100’000 queries per day limit, basically counting on the honesty of all users and spending considerable time to identify and contact administrators of query sources going way over the stated limit. Given a growing number of such abusive query sources, this manual approach does not scale well.

The “light” approach to enforcement also means that we err on the side of caution: we do only aggregate usage data from a selection of nameservers, and we only sample the data from the nameservers where we collect the usage data. Thus, we sincerely underestimate the actual usage by design.

Detecting and limiting abusive query rates

We are now taking additional action to ensure that our public nameserver infrastructure remains accessible for the tens of thousands of free users. These steps include:

• If nameservers can easily be linked to an organisation (rDNS exists and is non-generic, whois on the IP points to a promisingly identifiable organisation), we will send one notification to the abuse contact for the domain (using whois.abuse.net lookups).
• If no meaningful response is received to the above notification within a couple of days, the querying nameserver will receive a “REFUSED” return code instead of the actual DNS answer. Due to the way some DNS resolvers work, this may result in a higher query rate, since the resolver just tries it again and again to get an answer.
• If still no action is taken, and if the abusive rate of queries goes on for a longer time, the querying nameserver may get a “listed, hi” response.
• Those querying nameservers doing more than 1 mio queries per day for a longer time may get a “listed, hi” response.
• In counting the amount of queries, we aggregate networks belonging together (eg across multiple /24-sized networks, or having rDNS in the same domain etc).
• There are some organisations trying to play a “whack-a-mole” game by hopping between multiple public nameservers or similar techniques. We are not interested to play such games, but reserve the right to act accordingly.

Note that when a nameserver gets a “REFUSE” message from dnswl.org, it will likely get a similar response from other black- and whitelists as well, and the spamfilter will seriously underperform. It should therefore be in the administrators won interest to fix such a situation.

“listed, hi” response

In the extreme cases listed above, and for a limited time until query rates have gone down to the acceptable limits, we may return a special answer code to all queries, 127.0.10.3. The “10” indicates that this is a “special” return code, the “3” stands for “high trust” level.

This will cause that a spam filter will mark all mails as coming from a highly trusted server, and will thus result in some spams coming through to users.

While we do not appreciate having to cause such negative effects, the carelessness of the administrators concerned leaves us no other choice.

Out of the 50k to 60k nameserver IPs querying our public nameservers every day, less than 0.1% are affected by this stricter enforcement of our acceptable use limits. The number is even lower when considering the (estimated) number of organisations. However, some “big” nameserver providers are sometimes affected (eg Google public DNS).

What to do when affected by “listed, hi”

1. Contact us at admins@dnswl.org. As soon as we have a working administrative/operative contact, there is no reason to continue the “listed, hi” response. Note that while we are spread over multiple timezones and try to act fast on these issues, we do not have guaranteed 24×7 operations.
2. Switch away from a heavily used public DNS server, and use a local resolver.
3. If you are doing more than 100’000 queries per day, you must either get an rsync subscription, or stop using the dnswl.org public nameserver infrastructure.

[Update]

Some statistics on abusive query sources

It should be noted that DNS statistics are not always straightforward, so the numbers should be taken with a grain of salt. All our numbers therefore are heavily erring on the side of caution. Since we only collect and aggregate logs from a selection of our nameservers, and since we are only sampling the data (throwing away about a quarter of all collected logs), the real numbers are about three times as high as we report them.

All query sources which are deemed “abusive” are doing the high query rates for weeks and months. We may take a single day as an example, October 14th 2011, which happens to be a Friday (weekends generally have different patterns, but usually have the same Top N names).

Overall 78k unique IPs were querying the public nameservers on that day, each doing on average of 2’500 queries per day. The Top 100 Query Sources are all doing way above 100k queries each, the Top 20 Query Sources are way above 1 mio queries each. Some examples (aggregated by organisation, as far as this is possible based on rDNS):

Google DNS (8.8.8.8 etc) 33 mio queries (spread over ~ 50 IPs)
Dimenoc.com 6.7 mio (on a single IP)
Dyndns.com 5 mio (on 4 IPs)
Cogentco.com 4.5 mio
Safesecureweb.com 1.2 mio
Bluehost.com 750k queries (spread over 12 IPs)

Of those sources, only one (Dyndns) has promised action after being contacted.

[/Update]

The Goodmail accreditation service is shutting down, as numerous sources on Twitter and the Blogosphere are reporting.

If you are a current Goodmail customer and would like to register your IPs in the dnswl.org whitelist, please fill in the request form at http://www.dnswl.org/request.pl and write “Goodmail” in the comment box.

Sometime today, Nov 27 2010, amidst the hardware problems with one of our servers, we silently passed the milestone of 100’000 active entries in the dnswl.org database (it’s slightly more IP addresses, because there are also some ranges of IP addresses in our database). That data is used by about 50’000 organisations world-wide.

Based on our statistics, we cover about 90% of the volume of e-mail, and about two thirds of the number of IPs who send e-mail. We are still missing a “long tail” of about one third smaller mailservers in our database.

You may be curious to know how we arrive at these numbers. We obviously can not look into everyone’s e-mail logfiles, but we can look into the DNS traffic on our nameservers. There, we do not only look into who is querying our data, but we also look into what they are querying.

We aggregate this data on a daily and monthly basis to reduce the volume. We then filter out the “noise”: those with extremely low query volumes, those which are clearly dynamic/end-user space (“dynamic” in the hostname etc), and some other tests. The remaining IPs are added into a queue of IPs to be reviewed and assigned to appropriate DNSWL records.

From all IPs (including the filtered and those already assigned), we compute “magnitudes”. These magnitudes indicate basically the percentage of an IP from total world-wide e-mail traffic. Now, we do not directly measure e-mail traffic, but DNS lookups, from which we infer e-mail traffic based on the assumption that those with many DNS lookups are those with a lot of e-mail.

Given the caching mechanisms in DNS, our setup has a tendency to under-estimate the volume of the big senders. It’s a flaw we are willing to accept, especially as this effect is distributed over a very large number of IPs and so does not heavily distort the analysis.

Since an individual IP generally has a very low percentage of overall e-mail traffic, we would have very small numbers. We therefore use logarithmic magnitudes (see table at the bottom of this posting). All the “unassigned” IPs together are usually in the area of magnitude 9.0, ie about 10%. These 10% are (with some daily/weekly fluctuation) about 100’000 IPs. A considerable number of these IPs is later found to be snow-shoe spam or otherwise spammish and are thrown away, so we estimate that there are still between 50’000 and 75’000 IPs which we have not covered (ie total of 150’000 to 175’000 e-mail sending IPs).

Notes about our data

• We collect DNS usage data from six of our 14 mirrors. While it can be rather safely assumed that the large senders will be covered in any case, there may be regional differences which we do not account for (there is some regional bias in the geographical distribution of our nameservers).
• There is a considerable number of outright broken DNS queries to our nameservers. RFC1918 IPs, hostnames instead of IPs, Multicast IPs, IPs from 1.0.0.0/8 before it was being assigned – and most likely there are much more erroneous setups which we can not distinguish from regular traffic. We assume that this will not unduly distort our data in aggregated form.
• Some IPs are observed only for a short period of time. This may be typical snow-shoe behavior or other similar usage patterns. Strictly speaking, we should remove those IPs. We are however deliberately slow in removing them, because they may only be used sporadically (newsletters once a month, ...).
• We only started to collect full magnitude data a short while ago when we move the database to a dedicated, big, fat machine. The monthly magnitudes seem to be pretty stable, but the time before christmas may have unusual traffic patterns.
• We do not have such usage data for traffic between the big players in the e-mail space (eg between Yahoo and Hotmail) or internal to such and other organisations. Our data is based on the observation of DNSWL queries from the mostly small- to mid-size organisations that use our whitelisting information.

Magnitudes

As announced earlier, dnswl.org is changing it’s operating model. Users with high query volumes (> 100’000 queries/24 hours) and commercial filter vendors of anti-spam products and services are required to purchase a subscription (see here for full access requirements).

Over the next couple of days, we will start the enforcement of this model. “Enforcement” means that we may block your rsync access or your access to the DNS servers without further warning, since we usually do not know who you are.

If you use the rsync access today, please register early for the subscription. If you register before Nov 1st 2010, you will get an additional 3 months free with your 12 months subscription.

Subscribe today at https://subscription.dnswl.org/

The shiny new dnswl.org subscription management website is now in public beta.

Who could and should use this website?

All users who are expected to need a subscription (doing more than 100’000 queries/day on the public nameservers, commercial vendors of anti-spam solutions, rsync access for other reasons) can use the subscription mechanisms starting today. The site is still in beta mode, meaning that things may break. However, the actual rsync distribution mechanism is considered to be in production.

What is the benefit of using it at this early stage?

You get a longer transition period to the new rsync facility, and three months “early adopter” subscription for free (together with a regular subscription). Eventually, all current users of “rsync1.dnswl.org” will need to migrate. The earlier you start, the less you will have to rush towards the end. Enforcement of the new rules will start in about four weeks.

What does it cost?

We will publish the final pricing on www.dnswl.org as soon as we get out of beta. Current pricing is 120 Euros / 100 users, 300 Euros / 1’000 users, 1’200 Euros / for each 10’000 users. There is a reduced rate for education/academic institutions and not-for-profit organisations with a 50% discount.

For any questions, please contact us at office@dnswl.org.

Link to the dnswl.org subscription management website: https://subscription.dnswl.org/

dnswl.org has successfully provided whitelisting data for anti-spam filters since 2006. This is how we will ensure the continued success of dnswl.org:

The landscape for anti-spam solutions in 2011 is different from the landscape in 2006. dnswl.org must also adapt to these changes in order to remain a relevant player beyond 2011. Luckily, this will have little to no impact for most of our 50’000 users. We will provide the same independent level of data editing to live up to our promise:

Improve the reliability of the email system.

In a more mature and competitive market with big challenges ahead (adapting the anti-spam toolchain for IPv6), dnswl.org will also require a solid organisational and financial basis. However, dnswl.org has not and will not charge money to get on the list, as this would create adverse incentives.

We decided that dnswl.org will change to a subscription model for “heavy” users and sellers of anti-spam services or products. We consider those who do more than 100’000 queries per 24 hours on the public nameservers to be heavy users.

Prices have not yet been finalized. We plan to have a regular and a reduced rate; educational institutions, not-for-profits etc will qualify for a reduced rate. Those who contribute resources, data or time to the project will get free subscriptions.

For those who run a small- to medium-scale anti-spam solution (eg based on SpamAssassin) nothing will change, provided you stay below the 100’000 queries/24 hours limit. Also, we will not rush to cut someone’s access off just because he has a high-traffic day.

What else?

The financial basis for dnswl.org is not the only change that is needed to ensure the continued success of the project. We will need to broaden the reach of our editors, try to get more imports of existing whitelisting projects, and invest time and resources into improving our tools and internal data.

Getting ready for IPv6 and a somewhat improved infrastructure will be the first priorities as soon as the subscription model is in place.

Implementation of the subscription model

Migrating from our current model will be a challenge: We have no contact data for the current rsync users, and also not for the “heavy” users on the public nameservers. Further, the infrastructure and processes for handling the subscription needs yet to be built and tested. The introduction schedule looks roughly like this:

Please note that this schedule is subject to change depending on how things evolve. We will communicate regularly on the progress and next steps. You can always contact us at admins /at/ dnswl.org.

Contact for journalist enquiries: Matthias Leisi matthias@leisi.net, phone +41 79 207 31 08

Today, Spamhaus announced the public release of their whitelist. This is an important step for Spamhaus and for the concept of whitelisting in spamfiltering in general, and thus for us at dnswl.org as well.

With better whitelisting, the reliability of the overall email system can be greatly enhanced. Mails from senders with a good reputation are guaranteed to be delivered to it’s destination, and more stringent fitlers can be applied to mails from senders with unknown or outright bad reputation.

This concept has been at the core of dnswl.org in the almost four years since the start, and it has been understood and applied by our 50’000 users worldwide. We are glad that more players are increasing their efforts towards whitelisting, which will benefit all players in that area.

dnswl.org is aware that the landscape for spamfiltering (and thus for whitelisting) is changing. We have a number of steps planned to ensure that dnswl.org is fit to cope with this evolving landscape, and with the growing importance of whitelisting.

The three most important items on our roadmap are 1) to create a transparent and sustainable financing for our operation, 2) to encourage more partners to share their whitelisting data and 3) to add IPv6:

We will have a public announcement about the financial support for dnswl.org in a few weeks.

If you want to partner with dnswl.org, or share your whitelisting data with dnswl.org, or if you want to support our data editing, please contact us at admins@dnswl.org.

Very basic support for IPv6 is already included in the backend systems. However, there will be an intense testing phase before the first IPv6 addresses are published. dnswl.org will work with other players in the industry to ensure that the whole “toolchain” for DNSxL operations and use are available for a mixed IPv4/IPv6 world.

dnswl.org is now not only accepting donations via Paypal, but is also test-driving flattr.com, as you can see on the new icon on the right-hand side menu on the main website http://www.dnswl.org/

We don’t expect a lot of donations via Flattr, but it surely is an interesting approach to social payments.

Yes, we have been asked in the past why there is no “Donation” button on our website. Ask no more!

Donate to dnswl.org

Thanks a lot for all your support and donations.

— Matthias, for dnswl.org

At Swinog 19 (Swinog = Swiss Network Operators Group, meeting of Sept. 2009), Martin and Matthias did a presentation about dnswl.org. http://www.dnswl.org/files/dnswl-swinog19.pdf

Note that Swinog used to have their own whitelist. This whitelist was one of the sources on which dnswl.org started (and it was imported until the end of 2009). The Swinog whitelist is not actively maintained at the time of this writing (March 2010).