sa-update on OSX

Dear Lazyweb, how do I run "sa-update" on OSX?

I have SpamAssassin installed via "port install p5-mail-spamassassin". The only SA files appear to be in /opt/local/share/spamassassin/*.cf. The man page for sa-update says there should be a directory /opt/local/var/spamassassin/3.002000/ but nothing like that exists. How does this stuff work?

Tags: , , ,

9 Responses:

  1. mark242 says:

    - Run "spamassassin -D --lint" and look for a line like "dbg: config: using "/var/lib/spamassassin/3.002000" for sys rules pre files" (your path may vary)

    - "mkdir -p /var/lib/spamassassin"

    - "sa-update --updatedir /var/lib/spamassassin"

    That should place the files in the right spot. Also, as a bit of unsolicited advice, create a text file that looks like:

    updates.spamassassin.org
    70_sare_adult.cf.sare.sa-update.dostech.net
    70_sare_evilnum0.cf.sare.sa-update.dostech.net
    70_sare_genlsubj0.cf.sare.sa-update.dostech.net
    70_sare_header0.cf.sare.sa-update.dostech.net
    70_sare_html0.cf.sare.sa-update.dostech.net
    70_sare_html1.cf.sare.sa-update.dostech.net
    70_sare_obfu0.cf.sare.sa-update.dostech.net
    70_sare_oem.cf.sare.sa-update.dostech.net
    70_sare_random.cf.sare.sa-update.dostech.net
    70_sare_specific.cf.sare.sa-update.dostech.net
    70_sare_spoof.cf.sare.sa-update.dostech.net
    70_sare_stocks.cf.sare.sa-update.dostech.net
    70_sare_unsub.cf.sare.sa-update.dostech.net
    70_sare_uri0.cf.sare.sa-update.dostech.net
    72_sare_bml_post25x.cf.sare.sa-update.dostech.net
    72_sare_redirect_post3.0.0.cf.sare.sa-update.dostech.net
    99_sare_fraud_post25x.cf.sare.sa-update.dostech.net

    ...and run sa-update --channelfile /path/to/text/file --gpgkey (imported key). The SARE rules are great at boosting scores.

    • jwz says:

      I've got:

      dbg: config: using "/opt/local/etc/mail/spamassassin" for site rules pre files
      dbg: config: using "/opt/local/share/spamassassin" for sys rules pre files

      The .cf files are in the latter; the former contains the sa-update-keys/ directory, v310.pre, etc.

      So does this mean that whoever set up the macport decided to just leave the version number out of the directory name? If I run "sa-update" will it just update things in place? I thought there was a hierarchy of these things? I don't understand how any of this crap actually fits together.

      What is that SARE stuff? How does it differ from the default rules?

      • mark242 says:

        It looks like whoever did the port did a really crappy job of trying to fit the files into the filesystem hierarchy, but what can you do.

        The "site rules pre files" is the repository for "local" configuration. On a Red Hat box, this is /etc/mail/spamassassin. This directory shouldn't get touched by sa-update, but should be touched by you in order to configure your box. sa-update-keys is simply a store for your imported gpg keys (sa-update won't install external rule files unless you import a key from the source, or you force it).

        The "sys rules pre files" is the directory that gets modified by sa-update, and shouldn't be touched by you.

        SpamAssassin has one of the most convoluted configuration styles that I've ever seen. It looks at the "sys rules" directory, then it looks at the "site rules" directory (and you can override the config in the sys rules dir with a corresponding config in the site rules dir-- eg a "score BAD_ENC_HEADER 5.1" in any *.cf in site rules will override whatever score that test had in the sys rules dir). _Then_ it looks at individual user configurations to override whatever you put in the site rules dir. Yeah, it's that bad. (Luckily for me, this is hard to set up, so my antispam business does well.)

        SARE is a third-party set of rules that checks for, basically, more spam. A lot of the SARE rules have been incorporated into the SpamAssassin default ruleset, but a lot haven't. The SARE rules are a little bit more stringent, so they hit a few more nonspam messages, but in my experience the list above, scored correctly, doesn't cause any good e-mail to be dropped, and helps quite a bit over the default ruleset.

        • jmason says:

          First off:

          > sa-update --updatedir /var/lib/spamassassin

          Don't do that -- using "--updatedir" almost always isn't a good idea, as the man page notes. SpamAssassin will be looking for updates in the directory mentioned in the "sa-update" manual page, unless whoever did the MacOS port really broke things. (Let's hope they didn't)

          The "sys rules" directory, "/opt/local/share/spamassassin", is for the default, bundled ruleset. Once you run "sa-update" and it successfully downloads an updated ruleset, it creates a dir, typically something like "/opt/local/var/spamassassin/3.002000", which contains a newer version of that, and that will take priority as the new "sys rules" directory from then on.

          The key thing is, the latter dir will not be created *until* you run sa-update. That's why it doesn't exist yet. Once sa-update downloads a new ruleset, it'll put it into that dir, and any new SpamAssassin-using processes will look at that instead of the original ruleset.

          (The idea is to avoid overwriting a reasonable set of defaults when you download updates, and also to keep the writable stuff in /var and read-only stuff in /usr. That's good news for the FHS-using Linux packagers and Solaris systems, but probably looks a bit odd on BSD-based OSX.)

          If you run "sa-update -D", it'll spew lots of debug info -- including lines like this:

          [22417] dbg: channel: attempting channel updates.spamassassin.org
          [22417] dbg: channel: update directory /var/lib/spamassassin/3.003000/updates_spamassassin_org
          [22417] dbg: channel: channel cf file /var/lib/spamassassin/3.003000/updates_spamassassin_org.cf
          [22417] dbg: channel: channel pre file /var/lib/spamassassin/3.003000/updates_spamassassin_org.pre

          so you can see where it plans to put any news rulesets (if there are any).

          > SpamAssassin has one of the most convoluted configuration styles that I've
          > ever seen. It looks at the "sys rules" directory, then it looks at the "site
          > rules" directory (and you can override the config in the sys rules dir with a
          > corresponding config in the site rules dir-- eg a "score BAD_ENC_HEADER 5.1"
          > in any *.cf in site rules will override whatever score that test had in the
          > sys rules dir). _Then_ it looks at individual user configurations to override
          > whatever you put in the site rules dir. Yeah, it's that bad.

          that's not bad -- it's good! (well, I'm biased; I came up with it ;)

          • jmason says:

            oh yeah -- if you run "perldoc spamassassin" or "man spamassassin" there's a pretty good section explaining what the various config files do, and where it looks for them.

          • jwz says:

            Ok, after just running "sa-update" with no args, it did:

            [27144] dbg: channel: channel cf file /opt/local/var/3.002000/updates_spamassassin_org.cf

            which seems like a really stupid place to put the directory, but I guess if the other pieces of SA are looking there too, that's ok...

            • jmason says:

              Yeah, that should be the case.

              That path is like that because SA was built with a PREFIX of /opt/local, with no specific LOCALSTATEDIR specified, so that all its components would go into that subtree. Personally, I would have preferred if they used something under /var, myself, but that part is up to whoever made the package...

    • jwz says:

      So, I'm confused about where to put this SARE stuff on my system. I have a file, "/opt/local/var/3.002000/updates_spamassassin_org.cf", that begins with:

      # UPDATE version 556472
      include updates_spamassassin_org/10_default_prefs.cf
      include updates_spamassassin_org/20_advance_fee.cf

      Do I add those lines you posted to the end of that? Prefixed with "include"? Or does this go in another file somewhere else?

      • mark242 says:

        Don't add the lines to that file. Create a brand new file, anywhere on your filesystem (although I generally keep it in my local config dir-- for you, /opt/local/etc/mail/spamassassin. Call it "channels.txt" or something-- anything so long as it doesn't have the *.cf suffix. Put the lines in that text file, and make sure updates.spamassassin.org is the first line.

        Next, you need to import the SARE dostech GPG key.

        Then just call sa-update --channelfile /your/file/path.txt --gpgkey (dostech key). That should run succesfully. Stick that on a nightly cron job and you should be good.