The End of Spam as We Know It!!!...

[Guest Paracelsus]

From TechWeb Friday...Spam Problem To Be Solved In 24 Months

[epp_b]

Eh...wishful thinking, IMO. "Where there's a will there's a way"...or maybe I should say, "Where there's Mciro$oft, there's a way."

[Rons]

January 17, 2002"MS' highest priority must be security" – Bill GatesLOL

[nlinecomputers]

This can happen but only if all ISPs are aggressive about adopting SFP/SenderID. Most spam is spoofed to prevent tracebacks and allow virus hijacked zombie hosts to send it out. I say more like 48 months but it will happen as this WILL prevent spoofing and as ISP see the value it will be adopted. Also as long as email server software installs SenderID by default it will rapidly be installed in the Linux crowd as new distros replace installed servers. Sendmail is on board with this. This will also put a big dent in the phishing and the virus problem too.

[Cluttermagnet]

This can happen but only if all ISPs are agresive about adopting SFP/SenderID.  Most spam is spoofed to prevent tracebacks and allow virus hijacked zombie hosts to send it out.

Cool. I hope they do. The spam is getting real old, and filters have as many problems as they have benefits (I don't use them myself). Some folks are actually abandoning emailing- that's just incredible. It's "deja vu all over again" just the same as how fast-talking salesmen destroyed the utility of the common telephone. Let's hope they have a workable fix and get serious about saving email. Before it's too late.

[teacher]

Now will they be able to keep it off cell phones too? I was getting more spam on my cell phone (3 a week) than my computer so I had them turn off messaging to save me the hassle!Julia (Hopes pie in the sky dreams really do come true in this case)

[nlinecomputers]

I've been lucky they haven't found my cell phone yet and I can't turn off SMS messages as that is how I get paged by some of my clients.

[Grasshopper]

Phone spam???Ugh...I'm gonna have to take the pessimist route on this. It'll never go away. But if it actually does, the net as we know won't be as good. Not to say that spam makes it better, but to get rid of spam (viruses and other malware), you have to take away functionality...at least that's the bureaucratic way to do it. As long as money is changing hands, real freedom will not exist.

[nlinecomputers]

The only functionality taken away by SenderID is the abilty to spoof. If that is a desired function from you I'd like you to explain what useful feature you get from that. SenderID will harm no fuctionality of the net except automatic forwarding and that too can be fixed so that it works and still is traceable. http://spf.pobox.comAnd note you don't have to use it. You can still send mail without the new headers but keep in mind that users like myself will filter out all such mail as spam. If your unwilling to tell me where it comes from I'm unwilling to read it.

[epp_b]

So, from what I see, this basically authenticates the "FROM" header in an e-mail, right? I think that would work to a point. But what would be stopping a spammer from registering a whack-load of Hotmail, Yahoo and other free e-mail addresses? They can also still change the display name (assuming that also is not authenticated, which it is probably not), since most users are too...*ahem*...uneducated to take note of the "FROM" header anyway.

[nlinecomputers]

But what would be stopping a spammer from registering a whack-load of Hotmail, Yahoo and other free e-mail addresses?

Not a thing except that is already a solved problem. None of the freebie acounts will allow you to send much email at once. Most email filters, like spam assassin, already put more wieght against an free email account. And most free acounts limit the number of email addresses you can create or access from a single IP number. Try it and you will see. So it is very difficult to logon and spam accounts. Also the free acounts can track what IP numbers accessed them so if you did manage to spam via them they can still track you by IP number. There is a reason that spammers HAD to adopt spoofing. This is why.This isn't a single solution. It is designed to address one kind of problem. The problem of spoofing.From the SPF faq:

It doesn't really prevent spam. Spammers can always get throwaway domains, etc.Throwaway domains are the next step in the arms race. We can counter with: 1. fast automated blacklisting using spamtraps and attack detectors 2. simple reputation systems based on factors such as          * age of domain according to whois          * email profile of domain, eg. "too many unknown recipients"          * call-back tests to see if the sender domain is able to receive mail.       The reputation system can advise a receiving MTA to defer or reject. 3. legal methods following the paper trail of who paid for the domain. Here's an example of automated blacklisting in action: 1. A spammer spams.          * The spam comes from an SPF-conformant domain.                o That domain is on a widely published sender-domain blacklist.                      + The MTA rejects the message.                 o That domain is a throwaway, just-registered domain, and does not yet appear on blacklists.                  1. The spam gets accepted by unsophisticated MTAs which do not use other traffic-analysis methods to impose a crude reputation system on unrecognized senders.                  2. The spam also gets accepted by automated spamtraps.                  3. The spamtraps add the domain to the blacklist.                  4. (advanced) Some time later, the user checks email. Immediately before the display phase, the MUA re-tests the message against the blacklists, and discards it.                  5. Thanks to the greater level of sender accountability, lawsuits may begin against the spammers, and registrars may be subpoenaed for domain owner information. SPF strengthens administrative and legal methods.           * The spam comes from a non-SPF-conformant domain.                o Initially,                  1. Most legitimate mail will fall into this category.                  2. Normal content filters get to do their job.                  3. The usual false-positive/false-negative results apply.                 o Later,                      + Most legitimate mail will be SPF-conformant.                      + Some legitimate mail will not be SPF-conformant.                      + SPF-conformant receivers SHOULD receive non-conformant mail but MAY choose to perform additional filtering on it. 2. Eventually, as SMTP improves its immunity to spam, we hope spammers will get discouraged. If the volume of spam decreases, legal and administrative approaches become more effective; right now they are simply swamped. If there are only 10 spammers in the world, law enforcement can focus on catching each one. If there are 10,000 spammers, law enforcement throws up its hands, calls it a societal problem, and says it doesn't have enough resources to tackle it.    * The spam domain was registered with a domain registrar.    * If the registrar is cooperative, we can find out from the registrar who the spammer was; and the registrar can stop accepting their registrations.    * If the registrar is uncooperative, or if a spammer buys and runs a registrar, we can default-blacklist all their domains, in a political move similar to SPEWS's approach.    * Alternatively, since spam is becoming increasingly illegal, we can subpoena the registrar to find out who registered the domain, and sue the spammer directly.    * If the spammer registered the domain using false information, we can still go back to the credit card.    * If the credit card was stolen, that's a crime which can be addressed using traditional means.

[Cluttermagnet]

But what would be stopping a spammer from registering a whack-load of Hotmail, Yahoo and other free e-mail addresses?

Not a thing except that is already a solved problem. None of the freebie acounts will allow you to send much email at once.

Very interesting, Nathan-After reading your post, I also went to that SPF FAQ page you reference. Bookmarked. I will spend more time later today digesting the info there to the point of fully understanding it. Bottom line seems to be that "where there is a will, there is a way", and that way already exists in significant areas. We have not yet developed the will, I guess.

[nlinecomputers]

Cluttermag, one thing I've noticed is that 90% of my spam comes from spoofed addresses. If you close the huge hole in SMTP mail that does no cross checking you'd kill 90% of the spam and 95% of the virus activity on the net. It would be filtered out at the SMTP servers. The rest would be tracable which either is a legitimate advertiser or a stupid spammer who will be quickly filtered. It will not kill all spam but it will put a very large dent in it.Edit: I should add that most of the spam that you think is coming from hotmail or yahoo are spoofed addresses NOT real ones. This is why both M$ and Yahoo are big supporters of this. There name is being dragged into the mud by these spammers.

Edited June 7, 2004 by nlinecomputers

[nlinecomputers]

spammers will simply infect ligit smtp servers with agents that will forge headers and wait for requests about the forged email.  viola.  spoofed again.until tcp/ip v6 is ubiquitous, methinks the internet too full of holes to provide much in the way of universal security.

Mail servers aren't THAT easy to break in. If they were it would already be happening just to have a traceable host now.

[Ed_P]

In the interim maybe this will help.

[nlinecomputers]

I've tried something like that a few years ago. After having to get on the phone 3 times to my sister because she ignored the challenge and was unable to tell me her address over the phone(she was internet clueless) I bitcanned it. Too much work for people just trying to send mail. What if you have to fill out a form everytime you sent a post card the first time?

[ibe98765]

We've gone back and forth on this SPF issue in a few threads here over the past year. My contention was always that the spammers would find a way to beat this system. Apparently they have. So now we are back to using white lists with a fancy new name ("reputation analysis").I still believe that the only real solution to spam is going to be charging a small amount to send each email (first 200 or whatever a month are free).

Spammers Skirt IP Authentication Attempts  By Dennis Callaghan eWeekSeptember 6, 2004    As enterprises continue to register Sender Protection Framework records, hoping to thwart spam and phishing attacks, spammers are upping the ante in the war on spam and registering their own SPF records.E-mail security company MX Logic Inc. will report this week that 10 percent of all spam includes such SPF records, which are used to authenticate IP addresses of e-mail senders and stop spammers from forging return e-mail addresses. As a result, enterprises will need to increase their reliance on a form of white-listing called reputation analysis as a chief method of blocking spam.E-mail security appliance developer CipherTrust Inc., of Alpharetta, Ga., also last week released a study indicating that spammers are supporting SPF faster than legitimate e-mail senders, with 38 percent more spam messages registering SPF records than legitimate e-mail.The embrace of SPF by spammers means enterprises' adoption of the framework alone will not stop spam, which developers of the framework have long maintained.Enter reputation analysis. With the technology, authenticated spammers whose messages get through content filters would have reputation scores assigned to them based on the messages they send. Only senders with established reputations would be allowed to send mail to a user's in-box. Many anti-spam software developers already provide such automated reputation analysis services. MX Logic announced last week support for such services."There's no question SPF is being deployed by spammers," said Dave Anderson, CEO of messaging technology developer Sendmail Inc., in Emeryville, Calif."Companies have to stop making decisions about what to filter out and start making decisions about what to filter in based on who sent it," Anderson said.The success of reputation lists in organizations will ultimately depend on end users' reporting senders as spammers, Anderson said. "In the system we're building, the end user has the ultimate control," he said.Scott Chasin, chief technology officer of MX Logic, cautioned that authentication combined with reputation analysis services still won't be enough to stop spam. Chasin said anti-spam software vendors need to work together to form a reputation clearinghouse of good sending IP addresses, including those that have paid to be accredited as such."There is no central clearinghouse at this point to pull all the data that anti-spam vendors have together," said Chasin in Denver. "We're moving toward this central clearinghouse but have to get through authentication first."Link

[nlinecomputers]

People this is great. WE WANT SPAMMERS TO USE SPF. Don't you understand? If they use SPF then it is no longer joe jobbing. No longer forging email addresses. SPF != No spam. If they are using SPF then the email is traceable and I can very quickly setup blacklists that are valid. This stops the problem I've been having for years now of people sending out spam using MY domain name!"Reputation analysis" isn't new either. All the current anti-spam products are doing some kind of "reputation analysis". That is just a double-speak way of saying blacklists which will still always be needed and with SPF they will be much easier to maintain.

[ibe98765]

As I understood the article, I thought they were implying that the spammers were using fake addresses in their SPF registrations (and of course, changing them frequently)?

[nlinecomputers]

As I understood the article, I thought they were implying that the spammers were using fake addresses in their SPF registrations (and of course, changing them frequently)?

SPF wouldn't work if the address was fake. All SPF is a method to cross check what IP number(s) are allowed to send mail using your domain name. You can fake the name of a domain in email. It is IMPOSSIBLE to fake the IP numbers because they are not put on by the sender of email but by what servers the email passes through. Think of it like a postmark. You can put any fake address on a letter that you want but the postmark will always correctly tell you what post office first received the mail.To use the postal analogy SPF would be a public declaration by you that you will only send mail via the downtown post office. If mail with your return address is postmarked from the uptown post office then it is fake.Spammers are adopting SPF because they know if they ignore the protocol programs like Spam Assassin will weigh an email higher that has no SPF record. An email that fails SPF checking because it come from an incorrect IP address will certainly get weighed much higher.So as I said this is a good thing as this will make Spam much easier to track. It will shutdown the zombie Spam net that is being used to send Spam and put Spam back on a traceable path. It will make blacklists much easier to manage and be more accurate.

[ibe98765]

Can't you just keep changing your IP addresses?

[nlinecomputers]

Yes but such a change would have to take up to 3 days to work through the Internet. My mail server mail.n-linecomputers.com is hosted by myacen.com I would have to change providers to a new web host and that takes time and money. I'd have to rent a new webhost and change all my DNS records(SPF too!) to match my new webhosts server IP numbers. Once done I'm still just as traceable as I was before and if I am not then I get flagged as an SPF failure which makes programs like spamassassin much more likely to class my mail as spam.Change too often and your domain name will be blacklisted. And it wouldn't matter anyway. The domain name would STILL have to match the IP address you are currently using as a mailserver for SPF to flag you as good. The fact you are a moving target is irrelevant unless you fail to update your SPF records.Right now if you do have correct SPF records then programs like SA will not mark your mail as spam(though your mail may fail other tests that SA performs and still get marked spam.) If you have an SPF record and the mail doesn't check correctly against it then SA will pile high points against your mail. Most SA installs only take 5 points to be marked as a spam. I set mine to 4. Most mail has no SPF record at all so that is an unknown mark. Right now SA doesn't add any weight to unknown but as SPF grows in use the unknown weight will go up.

[ibe98765]

Here's another story on this from a couple of weeks back...

Spammers Hijack Sender ID Microsoft's E-mail-filter technology, Sender ID, is unpopular with open-source advocates but popular with spammers, who are using it to bypass other filters.By Thomas Claburn,  InformationWeek Sept. 9, 2004 URL: http://www.informationweek.com/story/showA...icleID=47102042 On the heels of the repudiation of Microsoft's Sender ID E-mail-authentication scheme last week by two major open-source software groups, spammers are doing the opposite: They're embracing the very standard intended to curb their abuses. According to E-mail security vendor MX Logic Inc., spammers are trying to make their messages appear more legitimate by adopting the Sender Policy Framework (SPF), which recently became part of Microsoft's Sender ID proposal. To comply with Sender ID, companies publish a list of authorized E-mail servers for the domains they control. That list is used by those receiving E-mail to make sure the purported server of origin matches the one listed in the message header. Because spammers may forge header information to disguise the origin of their messages, their spam would fail this test. But since spamming is legal, those spammers not engaged in phishing or other fraud may choose to accurately identify their mail servers to avoid filtering based on Sender ID compliance. And that seems to be what's happening. Based on a sample of 400,000 spam messages, MX Logic found that 16% had published SPF records. Scott Chasin, the company's chief technology officer, says this isn't unexpected. "The fact is that anybody can go out and purchase a $5 domain name and publish an SPF record," he says. "If you could publish your own credit report, how many folks out there would actually trust that?" "From the perspective of what SPF does, which is provide authentication to stop domain spoofing and phishing attacks, it's fantastic," he says. "To leverage it alone, as a spam solution, is not why it was created." He sees Sender ID as a vehicle for further industry innovation. "Conceptually, the industry needs to shift from identifying the bad senders to identifying the good ones," he says. Building upon authentication with reputation data is one way to do that. Not coincidentally, MX Logic last week added reputation analysis to its spam-detection scheme. Other anti-spam vendors, notably IronPort Systems Inc., have been touting reputation as an important component in E-mail filtering for some time now. In part, that's because they see it as a revenue stream. Marketers who spend the money required to comply with applicable laws and public expectation are generally willing to spend more money to have their legitimate E-mail pitches bypass filters. "A lot of the reputation-based services are indeed trying to position themselves as 'what you need to get delivered,'" says Ray Everett-Church, chief privacy officer at ePrivacy Group, a privacy consulting firm, and board member of the Coalition Against Unsolicited Commercial E-mail. He notes that while direct mailers might be interested in paying for such a service, there are other companies and institutions outside the E-mail-delivery-for-money business that still need to get their E-mail delivered. "There are other proposals out here that don't have the same PR budget that Microsoft has for Sender ID that don't depend on a certifying agency to control everything," he says. Such anti-spam proposals--under consideration by Marid, an Internet standards group--include Yahoo's DomainKeys and the Trusted E-mail Open Standard, which Everett-Church helped develop. These schemes rely on cryptographic verification rather than sender reputation. Sender ID's sudden popularity with spammers stands in contrast to its disrepute among some open-source organizations. Both the Apache Software Foundation and the Debian Project have objections to the terms of the current Microsoft Royalty-Free Sender ID Patent License Agreement. While a Microsoft spokesperson was not immediately available for comment, the company did offer a prepared statement: "AOL, Cloudmark, IronPort, VeriSign, Bell Canada, and the 54-member Email Service Provider Coalition have voiced support for the Sender ID license offered by Microsoft. There's broad support for Sender ID technology, and we encourage others to support and implement this technology so that together we can do more to tackle spam." Everett-Church suggests this disagreement will need to be resolved before Sender ID gains wide acceptance. "Until [there are] solutions are out there that are truly free to the market, open source, and based upon open standards, you're going to have a difficult time getting a lot of buy-in," he says. "At the end of the day, we don't want whole sections of the Internet dependent on a Microsoft licensing agreement. And that's not an anti-Microsoft thing. We wouldn't want it if it were all depending on IBM, Computer Associates, or Apple."

[Ed_P]

spammers were using fake addresses in their SPF registrations

What if the addresses referenced are business/street addresses rather than IP addresses? In registering the IP address is there any verification the information provided is accurate? Could someone register as:Me too123 Home StMyTown PA, 12345 ? So long as the registration fee is paid does the registra care? You can't sue a phantom.

[nlinecomputers]

Ed we are talking about IP addresses not phyiscal addresses and it isn't possible to fake an IP address and have SPF work. You've got to have a ligitimate mail server at a legitmate IP address to even do this. You can't fake this stuff. Otherwise the internet wouldn't work.

[Ed_P]

it isn't possible to fake an IP address and have SPF work.

I understand that but I'm not sure the article's reference to addresses is IP related in all cases. When you register an IP address aren't you required to provide information like your name and location? And is this information checked for accuracy?

[nlinecomputers]

This is my SPF record:

v=spf1 a mx include:sbcglobal.net -all

Lets break down what this means:v=spf1 This identifies the TXT record as an SPF string.a n-linecomputers.com's IP address is 69.93.68.34 (saturn.myacen.com).That server is allowed to send mail from n-linecomputers.com.mx This wizard found 2 names for the MX servers for n-linecomputers.com: n-linecomputers.com and saturn.myacen.com.(A single machine may go by more than one hostname. All of them are shown.)The servers behind those names are allowed to send mail from n-linecomputers.com.include:sbcglobal.net Any server allowed to send mail from sbcglobal.net is also allowed to send mail from n-linecomputers.com.-all No other servers are allowed to send mail from n-linecomputers.com.This is a good default.If you run BINDPaste this into your zone file:

n-linecomputers.com. IN TXT "v=spf1 a mx include:sbcglobal.net -all"

When a mail server sends a bounce message, it uses a null MAIL FROM: <>, and a HELO address that's supposed to be its own name. SPF will still operate, but in "degraded mode" by using the HELO domain name instead. Because this wizard can't tell which name your mail server uses in its HELO command, it lists all possible names, so there may be multiple lines shown below. If you know which hostname your mail server uses in its HELO command, you should pick out the appropriate entries and ignore the rest.So this should also appear in DNS. You may or may not be in charge of the DNS for these entries; if you are, add them.

saturn.myacen.com. IN TXT "v=spf1 a -all"

[nlinecomputers]

I understand that but I'm not sure the article's reference to addresses is IP related in all cases.  When you register an IP address aren't you required to provide information like your name and location?  And is this information checked for accuracy?

Yes you are required to tell them where you are and no it isn't checked much for accuracy. But when you register a domain you do have to pay for that service and that is checked via the credit card. But again none of this has to do with SPF. Sure you can lie all you want about your domain name. It still has to resolve to a real physical machine somewhere. Once knowledge of the spammers IP numbers are known he can be blacklisted.

[Ed_P]

Once knowledge of the spammers IP numbers are known he can be blacklisted.

But they can switch to another address. Could not a SPAMMER buy/register 10 addresses at one time? Costs less than a grand. He then sends out a million or two emails on address #1 on day 1. Does the same day 2 but now some are being blocked. Day 3, he sends out another million of two emails then switches to using address #2 and repeats the process for another 3 days. And so it continues for a month. At the end of a month he has sent out 30-60 million emails and it's costs him less than $1000.Would this senario not work?

[nlinecomputers]

Ed it not intended to stop that. SPF is only to stop joe-jobbing and spoofing. It is NOT a replacement for other forms of spam checking. As for throwaway domains, which is the method you describe, I will quote from the SPF faq.

SPF doesn't really STOP spam, does it?We've heard the complaints -- Spammers can always get throwaway domains, etc.At a high level, the answer is that we're moving from one paradigm to another: from "assumed innocent until proven guilty" to "assumed guilty unless proven innocent". The Aspen Framework brings two important tools to bear: reputation and accreditation. (A cartoon guide is available.)We agree that throwaway domains will be the next step in the arms race. We can counter with: 1. fast automated blacklisting using spamtraps and attack detectors 2. simple reputation systems based on factors such as          * age of domain according to whois          * email profile of domain, eg. "too many unknown recipients"          * call-back tests to see if the sender domain is able to receive mail.       The reputation system can advise a receiving MTA to defer or reject. 3. legal methods following the paper trail of who paid for the domain. Here's an example of automated blacklisting in action: 1. A spammer spams.          * The spam comes from an SPF-conformant domain.                o That domain is on a widely published sender-domain blacklist.                      + The MTA rejects the message.                 o That domain is a throwaway, just-registered domain, and does not yet appear on blacklists.                  1. The spam gets accepted by unsophisticated MTAs which do not use other traffic-analysis methods to impose a crude reputation system on unrecognized senders.                  2. The spam also gets accepted by automated spamtraps.                  3. The spamtraps add the domain to the blacklist.                  4. (advanced) Some time later, the user checks email. Immediately before the display phase, the MUA re-tests the message against the blacklists, and discards it.                  5. Thanks to the greater level of sender accountability, lawsuits may begin against the spammers, and registrars may be subpoenaed for domain owner information. SPF strengthens administrative and legal methods.           * The spam comes from a non-SPF-conformant domain.                o Initially,                  1. Most legitimate mail will fall into this category.                  2. Normal content filters get to do their job.                  3. The usual false-positive/false-negative results apply.                 o Later,                      + Most legitimate mail will be SPF-conformant.                      + Some legitimate mail will not be SPF-conformant.                      + SPF-conformant receivers SHOULD receive non-conformant mail but MAY choose to perform additional filtering on it. 2. Eventually, as SMTP improves its immunity to spam, we hope spammers will get discouraged. If the volume of spam decreases, legal and administrative approaches become more effective; right now they are simply swamped. If there are only 10 spammers in the world, law enforcement can focus on catching each one. If there are 10,000 spammers, law enforcement throws up its hands, calls it a societal problem, and says it doesn't have enough resources to tackle it.    * The spam domain was registered with a domain registrar.    * If the registrar is cooperative, we can find out from the registrar who the spammer was; and the registrar can stop accepting their registrations.    * If the registrar is uncooperative, or if a spammer buys and runs a registrar, we can default-blacklist all their domains, in a political move similar to SPEWS's approach.    * Alternatively, since spam is becoming increasingly illegal, we can subpoena the registrar to find out who registered the domain, and sue the spammer directly.    * If the spammer registered the domain using false information, we can still go back to the credit card.    * If the credit card was stolen, that's a crime which can be addressed using traditional means. (20040702) Scott Kitterman has posted a suggested refinement to the above plan.

The whole faq is at http://spf.pobox.com/faqYou guys should really RTFWS. Trust me there isn't anything you guys have thought of that we haven't covered.