|
Donation/Premium |
|
 |
|
|
|
|
|
|
|
|
|
| View previous topic :: View next topic |
| Author |
Message |
tembow
Blue Angel
Premium Member
Joined: Oct 10, 2005
Posts: 2945
|
 Posted: Sun Jun 22, 2008 11:05 am Post subject: Bulk Reporting Project Plan |
|
|
I have my dream.
A Bulk Reporting Project based on existing building blocks
(a) Suppose we extract all the spammed URLs by following several of the highest used name servers, every day, from their listings in the name server trackers
This makes database A
(b) Suppose we extract the URLs by following several heavily abused registrars, typically used by the same URLs as in (a) over the same period, from their listings in the registrar trackers
This makes database B
Theoretically
Database A will provide a wider scope of Registrars than Database B
Database B will produce a wider scope of spammed URLs than Database A
However, neither database has a measure of how heavily any site is spammed.
(c) By scanning the hourly and 24-hour lists at abusebutler.com, we could detect recent heavily spammed sites.
This makes database C
Data from A, B and C would be combined, and dupes eliminated.
All sites on client hold would be flagged as suspended.
All remaining sites would be identified with the brand, and the registrar if not already known.
Sites that do not load (eg name servers suspended, IPs blocked, time-out etc) would be flagged as unavailable.
By sorting the spam-branded and live sites into the sequence: name within brand within registrar, we have a bulk report ready to go to all registrars.
Either the request can go via email to each registrar,
or each registrar can be emailed a link to their respective section of the database, stored on a web site.
Periodically the database is verified, and status flags updated (suspended/client hold, suspended via name server removal, etc)
Registrars can have views of their own database section, able to be filtered by brand, by status flag etc. for different views. Each brand heading is a link to its spamtrackers criminal evidence page.
Registrars and general viewers can see the relative performance of all registrars for comparison purposes.
The various building blocks for this plan already exist.
1. rss.uribl.com and abusebutler provide the base data in the
2. IDSpam and similar (background) running tools can identify brand and registrar
3. tools exist to record the status of URL, and identify name servers per URL (traversal method from Complainterator)
4. Registrar contacts from Complainterator
5. A prototype implementation has already proven highly successful at the bulk spam reporting project
That leaves redirection target URLs for which there are detection tools, and high use IP addresses for reporting to ISPs.
That is the dream. It could happen.
|
|
| Back to top |
|
 |
AlphaCentauri
SIRT Handler
Premium Member
Joined: Nov 20, 2003
Posts: 2895
|
| Posted: Sun Jun 22, 2008 1:15 pm Post subject: |
|
|
It sounds good. I might add database D: domains that share the same heavily spammed nameservers AND share the same site content on fetching the html code -- in order to pick up recently registered domains before the spam starts if they are domains for known spam brands. e.g.:
*domain example1.com is being spammed
*its nameserver is ns.example2.com
*domain example3.com shares the same nameserver
*Fetch source code for example3.com.
*If it matches a known spam site source code template, it is included in the database.
*If it is different it is kicked out for human review to possibly add/adjust templates or to identify innocent brands sharing nameservers.
*If it fails to load, the tool continues to check periodically.
*If it fails to load over an extended period, it is again kicked out for human review to evaluate whether there is successful IP blocking, whether spammers are trying to undermine the tool with massive numbers of sham registrations, whatever.
|
|
| Back to top |
|
|
tembow
Blue Angel
Premium Member
Joined: Oct 10, 2005
Posts: 2945
|
| Posted: Sun Jun 22, 2008 9:40 pm Post subject: |
|
|
I agree with the concept of closing criminal domains irrespective of spamming, so long as they are not hijacked subdomains of innocent parties of course.
I am not so convinced about building in too many situations where the design involves human intervention on any large scale. Automation is the key to the design, with any reliance on manpower kept to a minimum.
|
|
| Back to top |
|
|
efa
Lieutenant
Joined: Aug 31, 2007
Posts: 166
Location: Italy
|
| Posted: Sun Jun 29, 2008 2:22 pm Post subject: |
|
|
I want to offer my knowledge on creating what need on the client side to develop this project.
The important thing for me are:
0 - full automation
1 - non-blocking GUI applet
2 - fast CLI engine inside (developed in C)
3 - be crossplatform (do not penalize Linux/Mac users)
4 - GNU/GPL license, the best for a collaborative project, to gain support from opensource community and sourceforge http://sourceforge.net/
Part of that idea are already in xComplaint, the Complainterator for Linux, and the next version in C/GTK+.
|
|
| Back to top |
|
|
AlphaCentauri
SIRT Handler
Premium Member
Joined: Nov 20, 2003
Posts: 2895
|
| Posted: Sun Jun 29, 2008 2:34 pm Post subject: |
|
|
| tembow wrote: | | I am not so convinced about building in too many situations where the design involves human intervention on any large scale. Automation is the key to the design, with any reliance on manpower kept to a minimum. |
My idea is that if spammers figure out what we're doing, they will try to shoot it down, and one way is to include innocuous sites on the same nameservers, then complain when they are shut down. If every time they tried it, it didn't work, I doubt they would persist.
|
|
| Back to top |
|
|
|
|
|
You can post new topics in this forum
You can reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum
|
Powered by phpBB © 2001 phpBB Group
|
Forum FAQ
Search
Usergroups
Profile
Login to check your private messages
Login
