Notes and Mail annoyances

Dear Lazyweb,

Why do my pages in the "Notes" app keep duplicating themselves? It seems like I get a duplicate page every time I edit one of them, but it's a Heisenbug because I can't make it happen at will. If I edit a note on my iPhone, wait a few minutes, edit it again on my iPad, and repeat, it does the right thing. And yet, it seems like every time I edit a note then put my device away, the next day I've got two of them (the pre-edit version and the new one). WTF?

Also, is there still no way for me to get push notifications of new email on my iPhone? My mail lives on an SSL IMAP server under my own control, as is right and proper. I am unwilling to CC all of my email to Apple, Google or any other third party. And yet, I would like to get a notification when new mail arrives, instead of having to manually poll. Am I still out of luck?

Previously, previously.

Tags: , ,

27 Responses:

  1. Tim says:

    I've also seen "Notes" notes duplicating themselves and otherwise behaving in strange ways. Sometimes iCloud syncing of notes is outright broken -- for a while I was unable to get changes to a note to propagate from a 10.8.2 Mac to my iPhone.

  2. Eric TF Bat says:

    I used to get the duplicate notes problem with Tomboy Notes, a Mono app in Linux (yeah I know, "now I've got three problems"). Best I could figure was there was some kind of time mismatch that made one of my computers think it was much older or younger than the others, but I never figured it out, and when I discovered that (a) Ubuntu had broken something so that the notes were no longer synching correctly to the cloud and (b) Tomboy Notes didn't have any means of telling me that the notes weren't synched, I gave up in disgust and went back to using Emacs and DropBox.

    This anecdote brought to you by the letter Q, the number 12.10, and the innate human capacity for pattern recognition that leads inevitably to irrelevant asides.

    • Ronald Pottol says:

      You (not JWZ) might find Evernote useful. There is a Linux (well, java, with linux as the main target, not from the Evernote people) app called NixNote, web, mobile device, Windows and OS X apps. It does do sync, and off line (well the Windows, OS X, NixNote, and most of the mobile apps (if you are a premium user). It is much more than a synced note pad.

  3. Daniel Parks says:

    Me too! I host my notes on my own IMAP server, as I imagine you do. It sounds like Tim is saying that it occurs on iCloud as well.

    One oddity about the bug is that some notes appear duplicated on certain devices but not on others — I often have ~10 duplicates on my iPhone, 3 duplicates on my iPad, and no duplicates on my Mac. There seem to be underlying files on the IMAP server for each note.

    Even more frustratingly, it sometimes gets confused about which note is which (mixing up the titles) and so it is easy to delete the wrong note.

    I also have a problem where notes sometimes loses the last little bit that I typed, though it may only occur with the Mac version. Presumably it doesn't finish syncing with the server before I close the application.

  4. Jeremy Wilson says:

    Based on my own research, the only way to get instant notification is to run Exchange. Polling once a minute is the next best option.

  5. Jon Konrath says:

    I've seen the notes duplication thing, but I'm using my google account to store them. I think it has something to do with multiple computers involved, like if I edit a note on my phone and then it's still "open" (the app is backgrounded, but not killed) and I open the same note on my Mac or iPad. Or something - it is hard to duplicate.

  6. Keith says:

    There's a project called Z-Push which implements MS's ActiveSync protocol and will backend to an IMAP server and do push mail. It works okay, but last time I played with it was over a year ago. It looks like it had a release two weeks ago. The downside, of course, is you're not setting up and maintaining an additional piece of software.

    http://z-push.sourceforge.net/

    • jwz says:

      This sounds like "instead of using IMAP, entrust your email to a GPL re-implementation of some proprietary MICROS~1 protocol from hell". This sounds like not the best idea. Even if the nearly-nonexistent documentation was written by a speaker of English. Which it is not.

      Are you actually using this?

      • Rob says:

        I've used it a couple years ago. Back then it was kinda buggy. Be prepared to do plenty of debugging... and I hope you like PHP, 'cause that's what awaits you! (It may have improved massively, but I ditched my iPhone in late 2009 so I haven't had a reason to try it since then.)

        • jwz says:

          "I have wasted everyone's time by recommending some piece of junk that I am certain does not solve the problem."

          Roger.

          • Rob says:

            I wasn't recommending it. I was confirming your "not the best idea" assessment.

          • Keith says:

            It works for me as a single user pretty well. The problem I have with the package is it didn't support a decent From line when used with multiple users when using an IMAP backend. It'll just show the e-mail address and not the friendly name at all.

            However, I agree with your assessment in regards of changing your trust from IMAP to a reverse engineered MS protocol.

      • gryazi says:

        IMAP has a 'push' protocol other than 'poll the IMAP server?'

        [Although it sounds like the point is more that Apple removed an option to poll the IMAP server given that AT&T doesn't want anyone to actually use data on their data network?]

        • jwz says:

          Yes, IMAP IDLE has existed essentially forever. However it requires an open socket, so Apple hates it.

          • Jim says:

            The win that ActiveSync has over IMAP IDLE is that ActiveSync can more reliably detect a lost connection such as might happen on a cellular network, and in ways TCP/IP may not be able to detect. When ActiveSync goes idle, a "let's meet back here in x minutes" negotiation takes place, and if either side doesn't get the callback it knows the network has vanished and to pick a smaller X next time.

  7. Calvin says:

    I used to have huge problems with duplicate calendar, bookmark, and address book entries. The solution was to turn off iCloud on every device but one (including osx), letting the OS delete the cloud data off each device. Then on the remaining device make sure everything was in a good state and then re-enable each device one by one.

  8. Calvin says:

    You can do the above per service. For example, turn of notes syncing in prefs on each machine.

  9. Waider says:

    Duplicates: had this while storing notes on my own IMAP server (dovecot with most, if not all, defaults, FWIW), don't seem to have it with iCloud. Possibly it's dependant on the server capabilities or something weird Apple does with them.

    Push notification: running OSX Server gives me this, albeit not wholly reliably. From debugging I've done when it doesn't work, it seems like this needs you to have an Apple-registered (not sure if this means issued, approved, or just logged with their services) certificate hooked up to A Thing which pushes notifications to Apple's notification gateway. A Thing would appear to be this: http://www.opensource.apple.com/source/dovecot/dovecot-239/dovecot.push-notify/daemon/push_notify.m (which I note contains a suitably jwz-baiting comment about tabs)

    • jwz says:

      And what am I supposed to do with that? Have you actually used it?

      • Waider says:

        Ah, sorry, I should've been clearer (and less glib, maybe).

        OS X Server provides an email config which does the push notification thing out of the box. The active component of this appears to be the above Thing, which, yes, I'm using by virtue of it being part of Apple's standard config. I would dig more right now except that I'm currently on the wrong end of a connectivity break from this particular server. My intent was to point you at something that might assist you in finding out the mechanism by which this works as I can't imagine "Install OS X server" is the answer you were looking for.

        (also it would seem that this should be supported in postfix rather than dovecot, which is now bugging me)

        • Waider says:

          Dug a little more into this. It works like this:
          - the push_notify thing consists of a client and a server, the latter being the source link above
          - the client is just a plugin for dovecot that talks to the server via a unix socket and says, "new mail for $user"
          - the server does a bunch of lookups (a config file that tells it where to send notifications, some details from opendirectory to do with the user) and then sends a XMPP message to another server.

          At this point it looks like you get into the "Needs a cert registered with Apple" space. As best I can tell from what I've seen in the console logs, the push_notify server in my case sends the notification to a local instance of notifyd (which IIRC was at some point a jabber server, and may still be), which in turn punts it to Apple's notification service gateway. This all looks to be done with XMPP messaging, and the certificate is what authorises my server to connect. Certs are registered & managed here: https://identity.apple.com/pushcert/ (requires an Apple ID) but even if you use this you'd still need to figure out the XMPP message format(s) required.

          Clearly the "easy" option here is to install OS X Server. I must confess, however, that I'm intrigued enough to see what else I can find out about this.

          On a side note I did find a mention elsewhere that when an iOS device received a "you've got mail" notification, that's all it is - no other data like sender or subject line is passed along. It's up to the iOS device to go ask the mail server for details.

  10. The only ways I've had push email working on the iPhone are via gmail (using the exchange protocol) and just for a test, via Yahoo mail (where they have some crazy proprietary protocol). I'm fairly certain there's no way to do it with just IMAP on iOS.