c

 

c

 
 


In the early 2000s, there were high hopes that we could solve spam with filtering techniques. Through
the years, we learned that spammers always find new ways to bypass or confuse the filters.

With the existing Internet, attack and defense are asymmetrical. The defenses are in the clear waiting
for attack. It is like the Maginot Line. The spammers and hackers are anonymous. They can test and
probe the defenses. They can get around the waiting defenses.

Spammers are not all the same. We have many brute force spammers that won͛t run around the
Maginot Line. We have dedicated smart spammers that test their solutions. RBL and filters are effective
against the brutes but not the smarties.

In this asymmetrical warfare, the defense must be mobile. We won͛t be able to stop new attacks. We
need to detect them and react quickly to them.

 


 
There are targeted attackers. These are not spammers. They understand their prey. They do not send
emails in bulk. We won͛t consider them in this paper.






 

 
1.| Spam must be bulk. If someone sends a small amount of infomercial emails, it does little harm. If
spammers are sending out bulk emails. We can try to detect them based on their bulk
characteristics.
a.| Spam is defined as unsolicited bulk emails.
2.| Human users can decide if something is spam or not.
a.| Spam is different than unwanted email or bounces.


 





 
1.| Ôur defenses must be fast. The whole solution must be an end-to-end agile solution.
a.| Try Value Stream Mapping in Lean Development.
2.| We need a team to be responsible for all aspects of the solution.
a.| Silo approach will introduce too much lags and wastes.


 


With these goals in mind, let͛s look into the lean solution model. We should not look for silver bullet. We
should look for solution model that will evolve as battles take place. The model takes care of the whole
solution chain. PATH based approach and Lean Development seem to fit the bill.





c
 

We can detect spam passively or actively. In passive detection, we analyze the statistical nature of the
customers͛ email traffics. In active detection, we solicit and analyze the content of the spam traffic.

0

   
How do we detect bulk? If we inspect the SMTP protocol stages, we can see that there are many
opportunities to detect bulk.

1.| HELÔ
a.| During connection, we can compile IP connection statistics.
b.| If the characteristic of the traffic changes, it may signal spam run. We can throttle the
connection attempts.
2.| MAIL FRÔM/RCPT TÔ
a.| We can check the stat between the sender/recipient.
b.| If the characteristic of the traffic changes, it may signal spam run. We can throttle the
connection attempts.
3.| DATA
a.| Filters can scan and classify the emails.
b.| If the characteristic of the classification changes, it may signal spam run. We can throttle
the connection attempts.

G   
ÿse spam traps to collect and analyze spam emails. We can monitor the spam traffic statistics. We can
analyze the spam to identify the sources of the spam.

  

End users can provide invaluable feedbacks to the system. For false negative, we will get many new
spam candidates that slip through our protections. For false positive, we can collect the rating results
and try to detect the common issues without sample.

° 



    




The new solution must support continue refinement and even wholesale changes. Depends on the spam
received, we may have to roll out new solution in unexpected ways.

Ône idea is to keep the filtering processing a black box with simple interface. When it is time to change,
we can do whole sale re-implementation. Ôptional parameters may be useful too. Ôld version uses old
parameter list. New version can use the optional parameters. If old version retires, we can remove the
retired parameters.
£
 


The process must be automated. Ônly the absolutely unavoidable things should be inspected by human.
The applications should work to make smart default and avoid asking the operators for input.

A system must be in place to monitor the updates. If any anomaly is detected, the violation candidates
are delivered directly for human inspection. There cannot be any delay.

The update monitor can be a special twin of the detection monitor.

£
  


If there are spam samples that are not classified properly, it should be directed to spam experts
automatically. When we say improperly, we mean there are contradictions related to the sample.

As time goes on, the unfamiliar should become familiar. New tools should be created to handle
unfamiliar spam.


 


-iven the outline above, here is one design for future anti-spam solution.