Do you run a virus and spam scanner and on all my email?
Today around 85% of all sent email is spam. This is actually a big problem in terms of load and processing time.
On all recent servers we run a powerful antispam system called Spamassassin which uses a combined score system from multiple types of checks to determine if a given message is spam. This successive system can let pass some spam in the beginning and block messages with the same patterns latter when a certain spam score has been reached. In certain very evident cases an email will be blocked immediately if it is over the spam score limit.
This successive approach is important to prevent "false positives", the blocking of legitime email messages, which happens with too untimely antispam systems.
It is advisable to scan all your incoming email yourself on your local computer too with the antispam and antivirus software of your choice.
On v-servers we don't install Spamassassin and Clamav by default as they require a lot of cpu cycles and memory.