Listserv update – I turned on “munging”

(Update:  See a followup message here about a step that you might take to try to get the listservs working for you again.)

For those who are following my travails trying to get ISPs to accept our listserv email messages now that we are on our dedicated server (original post and update) … this posting about “munging” may be of interest.  Others are invited to skip this posting as being even more geeky than usual. 

Any ISP faces the unenviable task of trying to protect its customers from spam email.  Smart people in the Internet community have devised several approaches by which a sender of an email message can attempt to reassure the ISP that the email message is non-spam.  One of the approaches has a side effect for listservs that forces listservs to do something called “munging”.  I have just now turned on “munging” for some of the listservs that we sponsor.  Maybe it will help a little bit with some of the ISPs that are wrongly blocking our listserv traffic.

“Munging” is defined (wikipedia article) as “a series of potentially destructive or irrevocable changes to a piece of data”.  The term has a connotation that what may be contemplated are “vague data transformation steps that are not yet clear to the speaker.”

Ways that a sender can try to reassure an ISP that an email message is legitimate include DKIM and SPF.  The idea of DKIM and SPF is that our server publishes information in our DNS records that tells the ISPs that they can trust emails from us having certain characteristics that match up.  The ISP can look at the “From:” line of an email message from us, and try to match it up with the IP address that our email message is coming from, and if they match, then maybe the ISP will be willing to trust that the email message is not spam.  

The problem comes when the ISP is skeptical of a listserv posting.  The posting, like any email message, has a “From:” line at the top of the message.  The default behavior of most listservs is to preserve the “From:” line of the person who posted the message to the listserv.  The challenge, when the ISP is busting our chops, is that the “From:” will be the email address of the person who did the posting.  That email address will contain a domain name.  That domain name will not come even remotely close to matching the domain name (“oppedahl-lists.com”) that is tied to our listserv system or to the IP address of our server. 

But the challenge is a more subtle one.  The thing is that the person who posted the message to the listserv has an email address with a domain name in it.  That person’s email address is in the “From:” field.  ISPs may look up any DKIM or SPF records that are advertised by the system operator for that domain name.

So let’s suppose a fellow named Bob with the Binford corporation posts a message to our listserv.  Bob’s email address is “bob@binford.com”.  The Binford corporation advertises DKIM and SPF records that tell the whole world how you can recognize that any particular email message is legitimately from the Binford corporation.  Like maybe they come from some particular IP address or have certain cryptographic secret numbers in the SMTP header of the email message.   The problem of course is that Bob’s posting to the listserv then passes through our server and our server passes it along to our thousand subscribers.  Or at least tries to pass it along to our thousand subscribers.  Many of whom have skeptical ISPs that look closely to see if the email came from an IP address that is known to be legitimate for Binford.  Of course our IP address is not at all known to be legitimate for Binford.

In an effort, then, to try to deal with the busting-our-chops ISPs, the designers of the software for the listservs provide an optional feature called “munging” that can be turned on.   With munging, the listserv software replaces the “From:” header address of the person who did the posting with the list’s posting address that is something at oppedahl-lists.com.  The idea is to mitigate issues stemming from the original From: domain’s DMARC (that is to say, DKIM or SPF) or similar policies.

This means that the things you see at the top of the listserv posting will look a bit different than they did before.  

This means that the consequences of “reply” or “reply to all” or “reply to list” might be a bit different than before.

This means that the filters that you laboriously set up to filter listserv postings into particular folders may now not work the way they did before.

Sorry about all of that.