For years, an unknown number of users behind TOR exit nodes have been abusing dnswl.org resources. Apparently for every email they receive, they query dnswl.org via the web, using the search interface open to all users. This causes considerable load on our webservers, the database servers, and our network infrastructure, sometimes causing significant delays for legitimate users.
As of Nov 30 2015, there is a slight change to the PGP key we use for signing the download files – there is no more expiration date on that key. Private key and fingerprint remain unchanged.
The updated key can be downloaded here.
Starting in early November 2014, we took some additional measures to enforce the query limit on our free nameserver infrastructure of 100’000 queries per 24 hours per organisation. As a result, some people may get a “blocked” result in their spamfilter setup.
dnswl.org has gradually improved the level of IPv6 support over the past months. The easy part is offering services over IPv6:
- Most public websites are accessible over IPv6 (since well over a year)
- Incoming and outgoing mails may pass through IPv6
- Nameservers for both the dnswl.org and list.dnswl.org zones have a healthy mix of IPv4 and IPv6 (and georedundancy etc)
- Monitoring adapted to include (hopefully) all services which are also offered over IPv6
- As a general rule, and wherever available, internal communication between dnswl.org servers happens over IPv6
This is a first attempt at a screencast to highlight how dnswl.org operates. Stay tuned for more to come (also with better video and audio quality).
Our previous method of enforcing limits caused some concern, both in public and private conversations. The main argument is that causing false negatives is not acceptable in principle, not even for cases of obvious abusive use.
We listened to these thoughts, and have now changed our approach. The criteria for blocking such abusive nameservers are still the same: repeated use way above the 100k / 24 hours limit and no response to reasonable attempts at contacting them. Also, most of the things in our previous news item referenced above are still valid.
We restrict the use of the public dnswl.org nameservers to 100’000 queries per day for all organisations using it for free. With this limitation, we want to keep the traffic for all public mirrors (some of which are donated) at an acceptable level (currently 100 to 200 GByte per month).
Sometime today, Nov 27 2010, amidst the hardware problems with one of our servers, we silently passed the milestone of 100’000 active entries in the dnswl.org database (it’s slightly more IP addresses, because there are also some ranges of IP addresses in our database). That data is used by about 50’000 organisations world-wide.
As announced earlier, dnswl.org is changing it’s operating model. Users with high query volumes (> 100’000 queries/24 hours) and commercial filter vendors of anti-spam products and services are required to purchase a subscription (see here for full access requirements).
dnswl.org has successfully provided whitelisting data for anti-spam filters since 2006. This is how we will ensure the continued success of dnswl.org: