<LJ-CUT text=" --More--( 8%) ">
Every now and then it just goes out to lunch for 20 seconds to a minute. Here's what usually happens: I wake up, start reading my mail, and after I've read about 20 messages, the next time I select a new folder, it just sits there looking like the folder has 0 messages in it. I go and do something else, and after a little while the messages show up. Meanwhile Activity Monitor says that Mail.app is at 85% CPU.
Any time I make some big change, like selecting a thousand bounce messages and marking them Junk, it uses so much CPU that it actually makes iTunes skip for 10+ minutes.
The "Activity Viewer" window is mostly useless; it shows certain activity, but is notably silent any time the load has gone through the roof and I open it up to see what the fuck is going on.
Somehow, my search results are full of all these "phantom" messages. E.g., 50% of the search hits are "real" messages, and 50% are messages with the same sender/subject/date, but they are marked as unread and the "mailbox" field is empty. Clicking on them says "The message from XYZ concerning "ABC" has not been downloaded from the server. You need to take this account online in order to download it."
So I guess somehow, when I imported my mail, a bunch of crap ended up in the search index. How can I clean it? "Rebuild Folder" on the folders in question doesn't fix it.
Maybe there's some way to tell Spotlight, "forget everything you know and start over"?
- Update: "mdutil -E /" nukes the Spotlight index and regenerates it. I guess in a few days I'll know if it helped.
Update 2: Nope, that didn't fix it. Interestingly, "Spotlight" from the menubar doesn't show these "ghost" messages, only searching from within Mail.app. So there must be some other index that Mail.app uses that needs to be regenerated?
Update 3: Quitting Mail.app, deleting "Library/Mail/Envelope Index", and letting it spend 4+ hours re-importing all my mail seemed to fix it. Yay.
Junk mail filtering hardly works at all. It gets like a 50% hit rate at best, and it seems to totally ignore all the hinting I give it. For example: message A comes in, I mark it as junk. The next day, there's a glitch on my mail server that causes it to re-deliver some old messages, including A. Mail.app still doesn't know that it's junk! A bit-for-bit identical message to one that I explicitly flagged!
I suppose I should just give up and maintain a list of banned subjects and senders like I did before, but I Want To Believe.
Is it possible that the Address Book is as utterly, mind-blowingly worthless as it appears to be? Like, it can't figure out that "Foo Bar <foo@bar>" and "Foo Q. Bar <foo@bar>" are the same person? I know that's tricky, given that they have the exact same email address! And drag-and-drop hardly works anywhere. And there's no way to re-order email addresses after you've added them. And and and. Did they have a summer intern write this thing? Hey, it's not like address books are important or popular or anything.
Join us now and loath the software. . .
Agreed. It's not the hardware's fault. The hardware is the innocent bystander here being used by the evil software companies. Hardware isn't evil, software is. This is why I am a hardware guy. This is why I hate software.
I have an antique email address which all the spammers know and love. Since Jul6, Mail.app has correctly identified 564 spams, incorrectly passed 5 spams to the inbox, and processed 15 legit emails correctly.
I bet that last number is way different for you. Maybe Mail.app gets stupid when you have to mark "this is not spam" a lot. Or I maybe I should stop speculating.
The number of spams arriving in my inbox has not decreased much with further training.
I'm imagining a clever fool who programmed it to check for perfect matches with marked-as-spam messages, only to render all matches imperfect by modifying the message by marking it as spam!
I'm unsurprised that a heuristic method doesn't regard the two identical mails as junk. I'd imagine if you flagged 20 identical messages, it's get the hint, but similarly, would get the same hint if there were 20 somewhat similar messages with the same characteristic. I don't know if it would be economical in 2005 to do fingerprint-style scanning too, the incidence of identical spam mails in the wild, from my samples, are practically nil.
Taking a complete stab in the dark, the pauses may well be mailbox contention between delivery and reading - do you tend to maintain big mailboxes (100s-1000s or mails) that it has to recalculate stuff for when it gets a new mail ? I know from my experience with other software that they don't expect most users to normally have more than a few dozen mails in a box, so don't think about the impact of a couple of thousand mails having to be re-read/index/filtered/frobbed.
Which is why I'm still a complete luddite and can't get away from mutt. But then, I can't train my fingers away from vim, either, so everyone should shun me.
Fuck, I'm so backwards I'm still using pine. Though, I do have it using vim as the editor. I was almost ready to switch to mutt before I changed that setting...
Of course, I look at jwz's trials and tribulations and think maybe I should just sit here, fat, dumb and happy in pine-land.
I have a shit-ton of large mailboxes, but I have also seen it go out to lunch when the current, previous, and previous-previous mailboxes all had <10 messages in them each.
Should you wish to clear out everything Spotlight knows about a volume, remove the .Spotlight-V100 at the root of the volume.
I typically did this from single-user mode (booting with cmd+s), but it should work wherever. After that, reboot, and you get to watch Spotlight churn for a few hours rebuilding it all.
sudo mdutil -E /works just as well.
You're right. I always forget they added that...
Thanks, I'm giving that a try. Will report back in ~3 days!
A little late, I know, but:
mdimport -f ~/Library/Mail
will force a rebuild the relevant Mail metadata without wasting time slogging through the entire drive.
Well, it slogged through the entire drive already and didn't help (see above).
The "Activity Viewer" window is mostly useless; it shows certain activity, but is notably silent any time the load has gone through the roof and I open it up to see what the fuck is going on.
Mail.app has some undocumented debug flags:
/Applications/Mail.app/Contents/MacOS/Mail -LogActivityOnPort 143 -LogSocketErrors YES
Replace "143" with your port du jour.
The last two cases where Mail.app was acting crazy, it was the IMAP servers fault. Your results may vary, see store for details.
... and if you really want to know WTF is going on, there's ktrace(8).
so I find that my mac in general just randomly takes little naps from 20-120secs...are you sure this is specific to mail.app? granted my mac is old and unhappy and has very little memory and I don't have tiger on it...
(however it is a laptop and I can surf the net from my damn couch which is squishy and I like it)
It's definitely Mail.app, because Safari is totally responsive during the periods when Mail is having its little lie-down.
Is it memory? maybe if you make your swap directory bigger? I think you can add a scsi drive and use a whole drive. Though you have that one piece mac don't you, so that won't work.
well look at it this way at least your on a game supported platform now...so you can get more zombies, even if you can't read your email as well as you'd like.
http://www.stubbsthezombie.com/trailers/trailer2.html
Nah, despite all the GUI froo-froo, Darwin isn't dumb enough to page out the currently running application.
In any case, if the machine were swapping, other apps wouldn't function either (and the context switch to bring another one forward would take everything to the shitter).
This is Mail.app getting stuck in some blocked on I/O syscall, but I'll be damned if I know what it is. I don't do GUI mail.
I don't do GUI mail.
yeah that's jamie's thing *cackle*
This would be why I responded to you, not to him, since I have nothing of value to add on the problems he's had with Mail.app. But he's getting emailed anyway, so I'm pissing him off anyway.
I had been told that the Address Book hasn't changed much since NeXT days, dunno if that is true....
But the address book is amazingly shitty, and based on the summer interns I have at my work, I'd say it actually looks like it was written by some technical document writer... :)
I'm pretty sure that it's filtering spam at those moments, mine does that too. I'm not sure why it take so much CPU to filter through small amounts (<50) of messages though.
Can't be - it performs spam filtering and rule checking when it first recieves the mail (which is why I often see messages flagged as spam in my normal mailboxes).
Mail 2 is incredibly slow, even with my Inbox being only around 3000 messages. Mail 1 may have been a bit dry on features, but I didn't consume upwards of three cups of coffee just checking my email.
Could it be because of the new emlx files, and some IO thrashing and plist parsing? Surely they must have thought about that (although, not as much as simply letting Spotlight Importers split files into meta-files in the index (or something similar)).
Others have mentioned it, but you might have missed it: Spam Sieve is the way to go. I'm at 79% after a week (upgraded to Tiger), but after another week or so of training I expect to be in the 90s. Full access to the corpus and rules list, too. It works as a plug-in to Mail.app so you probably need to wait a day before installing the latest-and-greatest updates from Software Update, but the author is responsive and it is very well supported.
Why not use "top" in the Terminal instead of Activity Viewer?
Because that offers zero advantages and has a crappier UI?
iPulse is a thing of joy. Sits in your dock and tells you what's going on in a really funky way.
I have yet to find a really good email application for Macintosh. Despite its problems, Evolution still works the best for me. I have 1.4 gigs of mail, and searches and virtual folders still come up quickly. Sure the junk mail filter is just as useless, but I have most of my filtering done server-side.
When my laptop was in the shop, I didn't even try to move my mail to the mac on my desk at work. I ended up moving my mail to my home computer and using ssh and X11 to forward the display to my work mac. I'd rather put up with 5 second latency per click than deal with Entourage or Mail.
Hopefully either thunderbird or Mail will improve.
This may help your search result problem.
1. Quit Mail.
2. mv ~/Library/Mail/Envelope\ Index ~
3. mv ~/Library/Mail/Envelope\ Index-journal ~
4. Restart Mail.
(The Envelope Index file is a SQLite db.)
If you do this, Mail will need to reimport all the messages, which can take a painfully long time.
That fixed it, thanks! It took ~4.5 hours to re-import it all, but that was a lot faster than the ~3 days it took to import it the first time (since it didn't have to sectionize every file this time).
While those are annoying, I find that the threading model that makes off-line reading completely transparent makes it worth it. I love the way I can move on- and off- network and "it just works".
I disagree with what you say about the address book being worthless. I loaded it up with a lot of my addresses and it seems to work really well for me. (and when I'm online, it accesses my LDAP server)
Yeah, I hate the way mail handles Addresses. I once received an email from a the support team at my hosting company, they use a general email address which goes to all of them but when one of them sends you a message it attaches their name to it. ie: I could get an email from Steve, and another from Jim, and while it would say the message is from Steve or Jim they'd have the same return address.
Anyway, I received an email from Steve, and after I responded to him I added said email address to my Address Book and gave it the name "Host Support."
Now, when I want to write a new email to the support team I open a new message and start typing "Host Support" into the To: field, but nothing comes up. NOTHING. If I type "Steve", though, it comes up right away. I DON'T HAVE ANYONE IN MY ADDRESS BOOK NAMED STEVE.
So frustrating.
Yeah, the address handling is infinitely crappy. Not the crappiest feature of the mac, because it's only a low-order infinity, but it's crappiness is nonetheless infinite. Who, for example is this fucking Julia in my address book? For years now, usually 3-10 times a day, I have sent email to my friend Julian. One day about 2 months ago typing ju in the To field suddenly began completing to julia@mumble.stanford.mumble.mumble. Who the fuck is she? I've never even heard of her, though I'm willing to concede that I could have replied-all to some long list of names that contained her. But this can only have happened once or twice. Months ago.
But the crappiest feature of the mac is probably the triple-threat failure that happens when you begin typing into the search bar at the top of a finder window. It (1) begins searching immediately for all documents whose name contains the first letter you type, and (2) it searches with such moronic agressiveness that the UI is locked out of the CPU while it searches for them, so it doesn't notice that you've already typed, actually, the fucking word you're looking for. Hell-fucking-lo. But no, this is not just 2nd-order infinite suckyness, because (3) by default it searches your entire hard drive, instead of the folder that's being viewed in the finder window. Extra demerits because this whole file-finding thing has obviously been fucked since 1977. Imagine a feature so sucky that the intensely obscure find . -name=mumble -print is actually superior. That's our mac file-finding facility.
http://junkmatcher.sf.net/
Have you seen JunkMatcher? The combination of it and Mail's built-in filter gives me very good results. It is also free. There's also SpamSieve, which is $25 and supposedly very good.
http://c-command.com/spamsieve/index
A few comments:
When Mail is hung and eating 85% CPU, get a sample: $ sample Mail 5. Even better, use Shark to get a time profile of the system. File a bug, and attach that information to the bug report.
Can you elaborate/clarify Like, it can't figure out that "Foo Bar" and "Foo Q. Bar " are the same person? ?
For example: my address book contains "Jonathan Xyz <xyz@example.com>".
However, this person sends mail as "j.g. xyz <xyz@example.com>".
When I click on the triangle next to the name in the From field, it gives me an "Add to Address Book" option instead of an "Open in Address Book" option, because it does not understand that those are the same person.
If I click "Add to Address Book", now I have entries for both ""Jonathan Xyz" and "j.g. xyz" in my address book, both with the same email address. But they're the same person! So I select them both and do "Card / Merge Selected Cards".
Now there's only one card "j.g. xyz" and the name "Jonathan Xyz" has completely vanished. I guess it picked the name that was alphabetically earlier. That card now has two email address fields with the same address in them.
So as far as I can tell, if someone doesn't spell their name exactly the same way every time, or if what's in your address book doesn't exactly match how they spell it in their email, the address book just totally fails to function.
What kind of sense does it make for the address book of a mail program to not key off of email addresses?
When I click on the triangle next to the name in the From field, it gives me an "Add to Address Book" option instead of an "Open in Address Book" option, because it does not understand that those are the same person.
I'm not seeing this behaviour in general. I can reproduce it when the case of the email in my address book doesn't match the case of the email in the message (i.e. there are capital letters). This is a bug.
Now there's only one card "j.g. xyz" and the name "Jonathan Xyz" has completely vanished.
The non-matching info, such as this, gets appended to the notes field.
I agree there should be a better heuristic for matching names and resolving conflicts.
Well, in this case there were no capital letters in either email address -- the addresses matched exactly.
So is the way to tell Mail.app "this person has two names" to set the Notes field to "OtherFirstName\nOtherLastName"? Or does Mail not consult that, and that's just what happens when you merge two cards?
Well, in this case there were no capital letters in either email address -- the addresses matched exactly.
Hmm. Email me.
So is the way to tell Mail.app "this person has two names" to set the Notes field to "OtherFirstName\nOtherLastName"? Or does Mail not consult that, and that's just what happens when you merge two card
A person can't have two names, according to Address Book, the way they can have two phone numbers.
The notes field appending business is just a convenience so the duplicate name isn't deleted entirely.
Mail should key off the email address, not the name. Then, this wouldn't be a problem.
http://www.macdevcenter.com/pub/a/mac/2004/05/18/spam_pt2.html
" The Apple data kit allows the user to find the single document that best represents each topic. Best of all, it also produces a summary of a document. That's what allows the accompanying AppleScripts to write summaries of your reports (this is called Summarize, located in the Services menu for Mail.app). "
so did you look at the summary for the email? does that actually tell you anything useful?
Services/Summarize seems to always be grayed out.
I basically gave up on Mail.app's junk mail filtering and just use SpamAssassin on my mail server now.
Thanks very much - I have the same problem with phantom search messages so I will have to try your fix.
My pet hate is that there doesn't seem to be a way to sort search results by date by default - they always sort by rank initially. Usually I have a good idea of about when the message I'm looking for was so I find it much clearer if the results are sorted by date. Anyone know if it's possible to set this, perhaps with an addon?
THanks
Just found this thread after finding your other (much more useful to me) post on Mail.app flags values, which interests me as I'd rather not re-download GBs of mail just because Mail.app has marked thousands of my old messages as spam when I didn't disable junk message support before visiting the folders (disabling it later leaves the messages marked as spam). On another Mac where I downloaded after disabling it, all my messages have bits 29,25,24 set to 0. So I'll probably just reset those bits in all the emlx files and see how I go getting rid of the annoying brown message coloring without clicking them one by one :-)
Anyway: One note here - if you're using FileVault, that mdutil command wouldn't help, as you'd have to do 'mdutil -E ~' instead (your home directory has its own Spotlight index).
Although, 2 years have passed, so you've hopefully solved the issue if it did re-occur :-)