Fighting spam email — guilt by association

(Update:  our listserv server got migrated on about November 5, 2023 from an old IP address of 162.255.116.157 to a new IP address of 66.29.132.148.  As of November 10, 2023, the UCEProtect-Network lists both of our dedicated IP addresses 63.250.38.181 and 66.29.132.148 in its UCEPROTECTL3 category.)

Yesterday our firm came face-to-face with one of the ways that email system administrators fight spam — a very interesting guilt-by-association system called UCEProtect-Network.  This system collects spam reports and carries out a “cluster analysis” (Wikipedia article), aggregating the reports by groups of IP addresses from which the spam emails originated.  The practical result is that everybody who uses Microsoft to host their inbound email has stopped receiving any email from our firm (“oppedahl.com”) or from our listservs (“oppedahl-lists.com”).  This is not because either of our IP addresses has ever been the source of spam (neither IP address has ever been a source of spam) but because other IP addresses that are “nearby” to our IP addresses have been the source of spam. 

The server that our firm uses for our firm’s own email (“oppedahl.com”) is at IP address 63.250.38.181.  It is a “dedicated IP address” meaning that nobody else gets to use that IP address other than our firm’s server. The UCEProtect system has apparently found that there has been lots of spam originating from IP addresses like 63.250.38.x where x is something other than 181.  (In geek terms this is called “the 63.250.38.0/24 subnet”.)  This prompts the UCEProtect system to place our IP address on a “bad list” called “UCEPROTECT-Level2”.  Microsoft (and probably quite a few other email system administrators) chooses to block emails originating from IP addresses that are listed on the UCEPROTECT-Level2 bad list.  (Update:  as of 2023, the UCEProtect-Network has downgraded its disapproval of our IP address 63.250.38.181 from its UCEPROTECT-Level2 to the lower disapproval level of UCEPROTECT-Level3.)

The idea of course is that our firm will eventually catch on that our emails are getting blocked by Microsoft, and that we will eventually research it and find out that the blocking is happening because of this “guilt by association”, and that we will complain to the company that we pay to do our server hosting.  (In our case that is Namecheap.)  The situation, of course, is that the 63.250.38.0/24 subnet is “owned” by Namecheap, and they “rent” our IP address to us and they “rent” the other 254 IP addresses in the subnet to 254 other customers.  And the idea is that we will complain to Namecheap and that this will prompt Namecheap to look closely at the outbound email traffic from those 254 other Namecheap customers.

It’s probably actually a lot more than 254 customers, because at least one of the IP addresses in that subnet is probably used on a “shared hosting” machine, which contains dozens of “virtual machines”, each of which is rented to an individual customer, any one of which might possibly be a spammer.

What tipped me off was that we started seeing bounce messages from email addresses like “x@hotmail.com” and “x@msn.com” and “x@live.com” and “x@outlook.com”.  But of course there are lots of businesses who pay Microsoft to host their own email that uses their own domain name (through the “outlook” email system) and those messages have also been getting bounced.  So I researched this, learned about the UCEProtect listing of our IP address for our server that sends “oppedahl.com” emails, and opened a trouble ticket with Namecheap.

I have to imagine that what is going on now is that the folks at Namecheap are running down the list of every customer in the 63.250.38.0/24 subnet, one by one, trying to figure out which customer is the cause of the UCEProtect blockage.  The blockage is blocking email to every Microsoft destination from every legitimate Namecheap customer whose outbound email server has the bad luck to be in this subnet.

That’s the server that our firm uses to send our “oppedahl.com” email.  But we also host a couple of dozen listservs — email discussion groups for intellectual property professionals.  (You can see some of them here.)  These listservs are hosted on a separate hosting server with a dedicated IP address of 162.255.116.157.  (Update:  as mentioned above, this dedicated IP address got changed on about November 5, 2023 to 66.29.132.148.)  And the UCEProtect system has also placed this IP address on a “bad list” called UCEPROTECT-Level3.  (Recall that the previous bad list was called “Level2”.)  Level3 is different from Level2, and in some ways is worse, it turns out.

When the UCEProtect system does its cluster analysis of spam reports, one of the ways that it aggregates the reports is to group the reports by “subnet”, that is, grouping the subnets like 63.250.38.x.  That grouping is what gets an IP address on a “Level2” bad list.  But the other way that the UCEProtect system groups spam reports is by something called ASNs (Autonomous System Numbers (Wikipedia article)).  Oversimplifying slightly, an ASN is a group of IP subnets that are all owned or controlled by a single entity.

It turns out that our IP address 162.255.116.157 is within ASN 22612 which is a group of 43264 IP addresses that are owned and controlled by Namecheap, and according to the UCEProtect system, 69 of those IP addresses have been emitting lots of spam recently.  This has prompted the UCEProtect system to place all of those 43264 IP addresses onto its “Level3” bad list.

If there were ever a real-life example of “guilt by association”, this is it.

So anyway of course I opened a trouble ticket with Namecheap about this problem as well.  And I have to imagine that the Namecheap folks are running down the list of all of its customers who are on those 43264 IP addresses, to try to figure out who the 69 spammers are.

I am a system administrator.  This is making trouble for me.  Right now, nobody at my firm is able to send email to anybody whose email is hosted by Microsoft.  I suppose some people in my position might get mad at Microsoft.  But it is sort of hard to really blame Microsoft for choosing to use every system at its disposal to try to block spam.  And I suppose some people in my position might get mad at the people who run the UCEProtect system.  But it is sort of hard to really blame the people at the UCEProtect system for choosing to carry out these two distinct kinds of cluster analysis to try to put pressure on hosting companies like Namecheap to try to block spam that originates from their own customers.