Is programming an Arduino from MacOS always a nightmarish clusterfuck, or am I just lucky?

I spend like 95% of my time wondering why the god damned serial port isn't working.

Of late, it goes like this:

  • Upload the code.
  • Unplug the FTDI from the Arduino Ethernet and change something.
  • Plug it back in.
  • FFS, where did /dev/cu.usbserial-WTFBBQ go?
  • Yup. It's gone.
  • Power cycle my USB hub.
  • Oh great, now my Mac's entire USB stack has gone away and now I don't even have a mouse or keyboard.
  • Hold down the power button and reboot.

Unplugging the FTDI doesn't always make /dev/cu go away, just like nine times out of ten. But as soon as that happens, I'm within minutes of needing to hold down the power button.

To whom do I address my hate? The authors of the comically horrible Arduino IDE app? I'm gonna guess, "yes, there" because the fucking thing is written in Java and anyone who made that decision any later than 1997 clearly can't be trusted to find their own ass in the dark with both hands. But I would certainly entertain the idea that this is all Apple's fault somehow. Maybe they've assigned maintenance of their USB stack to some intern who sees it as a great learning experience.

But, holy crap, do people really put up with this? Or am I just lucky? Or doing something wrong?

Is someone going to say, "You just unplugged it? You can't just unplug it, USB isn't hot-swappable! Chickens must be bled first!" Ok, but I didn't read anything in the manual about these chickens, tell me more.

Tags: , , ,

55 Responses:

  1. dr2chase says:

    I had much better luck with Adafruit's USBtinyISP and the 6-pin programmer.
    But I don't usually do Arduino.
    But-but, whatever I tried before USBtinyISP, sucked a lot like you describe.

  2. Gabe Ortiz says:

    Have you tried a USB 2.0 hub? The 3.0 ports on the 2013 and later MBPs don't play nice with Arduino consistently.

    • Ingmar says:

      Or anything. I have to plug a USB 2.0 cable into my USB 3.0 external drive so OS X doesn't panic after a few hundred MB transfers.

      I suspect the real reason Apple is removing USB ports is because they could never get them to work right.

  3. Yuma Tripp says:

    Best use of Leon evaaaar!

  4. Lorin says:

    It has been a lot less hassle for me but I have always used a USB Arduino Uno directly connected to the Mac. I have not had similar issues across a bunch of different Mac models. The tty disappears for me sometimes, but unplugging the Arduino and plugging it back in, and then re-selecting port in the IDE is all that is required to get it working again. I have not had to reboot my mac or had any similar issues.

    The Arduino 'IDE' definitely sucks, but the USB hub could be part of the issue, I'm guessing you've already tried connecting directly to the Mac, so it's also possible the FTDI or Ethernet Arduino suck too. The Arduino + ethernet shield worked well for me, definitely bulkier though.

  5. CBGoodBuddy says:

    I've been programming regular Arduino Unos (not ethernet Unos), and haven't really had a problem. Maybe one time out of eight, the USB serial device doesn't recognize and I physically un-plug and re-plug to get it to work. FWIW, I plug the Arduino directly in to a MacBook Pro USB port, and not a hub.

    • jwz says:

      Well, since I have had Arduinos completely destroy the hub they're plugged into before, there's no way it goes directly into the computer without a condom.

      • CBGoodBuddy says:

        Opto-isolated USB pass-through devices...
        http://a.co/2e2sCQf
        http://a.co/4HkNRil
        ... not saying this will work but it would isolate USB stack issues from power issues.

        • Andrew Klossner says:

          Those two gadgets support only USB 1.1 speeds, 12 Mb/sec. I see a USB 2.0 isolator listed on Amazon (dp/B01E68ME7S) but it costs $190.

          • CBGoodBuddy says:

            12 Mb/sec sounds perfect for upload and debugging Arduinos.

            • dan mcanulty says:

              Yeah, totally adequate, I think that's technically 'full speed', which should be no problem.

              Bigger issue could be that FTDI drivers are really hand-holdy. In general trying to reliably connect to usb serial devices is something I struggle with on a regular basis. I'm not fully following the issues and hardware described in the blog post, but my experience with arduinos is that there's a sweet spot that you find where things work reliably, but if you're struggling with other issues beyond the usb serial drivers, the combination of the two could be really confusing.

              But assuming there was a significant amount of time between frustrations beginning to blog post time, there might be some benefit in finding an old laptop you can wipe clean and installing from there.

              Arduino IDE is perennially frustrating, I really enjoyed discovering Platform IO IDE last time I had to write arduino code. It solves a lot of the problems that used to drive me nuts. Generally I try to stick with Arduino Uno, it took me a while before I switched from Diecimila, one of the reasons being actually because they switched from using an FTDI chip to using a usb-serial interface that they rolled themselves, and I had gotten familiar with the quirks of the FTDI driver. I don't think FTDI has changed much over the years, but I'm surprised you're using it with whatever board you have because I thought they left them behind for good years ago.

              In case it's of any use at any point, Keyspan continues to make what seems to me to be the most reliable Mac USB-Serial device and drivers. They're not without their headaches, but if anybody is weighing the different devices against each other for mac programming, Keyspan is still the most consistent in my experience.

              • dan mcanulty says:

                Oh, I re-read the blog post after thinking and metabolizing a little bit, and figured out where the FTDI is coming in, as an external cable.

                Okay, so my experience with Arduino Ethernet was that I bought one about five years ago when I was super broke and the outlay was a considerable mental investment and it was so difficult to use that I literally never used it and it's still mouldering in a box.

                Arduino Ethernet, literally the least useful Arduino I've ever worked with. I thought they had discontinued them and saved everybody the hassle.

    • Lorin says:

      same.

  6. relaxing says:

    I'm inclined to blame Future Technology Devices, Inc. for their shoddy USB-serial drivers and hardware.

    I find modern Arduinos with Atmel USB hardware have been stable on Retina MacBooks.

    My students with Windows laptops seem to have much more trouble.

    • deepatron9000 says:

      Any evidence for blaming FTDI? Many devices use FTDI hardware and software. Windows and how it deals with drivers sucks. Remove whatever windows thinks you should use for an arduino and install manually. Works fine, every time.

      • jhitesma says:

        Well FTDI is known for playing somewhat nasty tricks in their drivers to try and stop clones: http://hackaday.com/2016/02/01/ftdi-drivers-break-fake-chips-again/

        Ideally a converter bought from somewhere like sparkfun would have a legit chip...but the FTDI chip is so ridiculously counterfeited now that even reputable suppliers have wound up distributing counterfeit without realizing it. Bunnie has a number of interesting articles about counterfeits (of various chips and memory devices) he's run into even from reputable vendors...just one that comes to mind as a good starting point: https://www.bunniestudios.com/blog/?p=2037

        If Jamie's adapter is the sparkfun one linked (And I assume it is...why else would he link to a discontinued one) then chances are it is a legit device. But there's always the chance it may not be and FTDI's driver bombs are contributing to the issues.

        In this case it would almost be nicer if they were still bricking them so at least then it would be obvious if he's got a legit chip or not and could quickly/easily rule that possibility out.

  7. Jan Kujawa says:

    I think this is mostly FTDI's fault.
    But all embedded development is awful. Bootloaders that don't. JTAGs playing hide and seek. Customer support people named Nordic Aryan.

  8. Brian Carnes says:

    You're doing nothing wrong. This is, in fact, a thing. It happens to me often during embedded development on a MBP w/ non-Arduino targets.

    I encounter similar symptoms (unable to use built-in USB-connected keyboard or trackpad, ditto for external kb/mouse, so power cycle via long power button press) when unplugging USB serial cables in, oh, maybe 4% of the time, on a MBP that has never seen anything from Arduino.

    As a very small consolation, every once in a while waiting a few seconds, and plugging it again once or twice MAY revive your mac's USB stack and ability to use keyboard/mouse.

    Now that you have another datapoint, we can mull over: the "comically horrible" Arduino IDE is written in java, so really shouldn't be able to wedge your mac's kernel behavior in this way.

    Blame the Apple USB stack or perhaps whichever kernel object holds the FTDI driver. Even if you've got one of the counterfeit FTDI chipsets in your cable, it should not be able to wedge your Mac's USB subsystem in this fashion.

    > do people really put up with this?

    No: once I get unlucky on my Mac, I kick myself for forgetting it has this Achilles' heel, and after I power cycle switch over to a Linux laptop or VM(*) for remaining things that need to get done via serial that day.

    If it helps: I have distinctly noticed the kb/trackpad going dead right upon the unplug of the serial cable from the MBP built-in USB ports. (then the replug sometimes revives the USB bus). Never noticed the moment of plug-in triggering the USB death.

    (*) Ensuring the USB/serial cable is assigned to a Linux VM running inside VMWare Fusion seems to prevent the Mac host from blowing chunks when you unplug. The mac host is then merely passing raw usb info into the VM, using vmware's drivers to do so, where the linux ftdi driver handles it from there. So fingers seem to point at the usb-serial/ftdi drivers in MacOS...

    • Russ says:

      Yup, I've had more than a dozen FTDI chips plugged into a combination of USB hubs and hotplugged in every way possible on Linux. Never a single problem. I've even severely abused the stack with a bitbanged microcontroller talking USB (v-usb). Again, never a single problem.

  9. John Adams says:

    I don't have this problem with FTDI on Olimex or FTDI on Arduino. Upgrade your FTDI drivers?

  10. Kaleberg says:

    It sounds like bad luck to me. I've been programming all sorts of Arduinos from a Macbook Pro over the past few years, and the USB stack seems to stay in place. I've had the IDE get confused when I'm programming two or three Arduinos in different categories, but that's my own damned fault.

    I suppose you could spring for a Yun. You can program them over WiFi so there's no USB stack to deal with. Still, spendy$

    • jwz says:

      It's hard to accept that "this is 100% reproducible" is "bad luck".

      • A Kaleberg says:

        I guess I've been lucky. It could be something I do, but I can't imagine what, so I'll chalk it up to luck. I'd give you the old pep-talk and tell you to download a new FTDI driver or start explicitly unloading and reloading the FTDI kext, but that is just likely to cause fresh hell. I tried playing around with kernel extensions once to drive a Polulu servo controller directly from my Mac with libusb, but whatever Apple put in there was way tenacious. It just laughed at sudo and lied like the devil himself.

      • dan mcanulty says:

        I'm just going to repeat this here, in the even that it might otherwise be missed, and a search of all the comments implies that nobody else is specifically talking about arduino ethernet. My experience may be years out of date, so could be inapplicable now if they've redesigned it, but in my experience arduino ethernet sucks and should be avoided. I thought they had discontinued them and I considered that to have been a smart decision that would save a lot of people a lot of unnecessary misery. Regular arduino works just fine, and ftdi itself is not an order of magnitude more of a time sink than any other mac usb serial driver. The issue for me was that arduino ethernet was excessively complicated and not a good choice for an embedded networked device. At the time anyway, and it sounds like possibly still.

        If I wanted to do something with embedded ethernet and I wanted to do the least amount of work possible, I would probably first try for a raspberry pi hosting an arduino. In my mind it's the best of both worlds (the arduino GPIO pins are very fault tolerant, unlike the RPi's) and it costs about the same as an arduino ethernet.

  11. Leigh Klotz says:

    Pretty sure this is the FTDI driver problem. I didn't see this on other Arduinos. I am making lots of progress with the new ESP8266 devices, such as NodeMCU and Wemos with the Arduino IDE.

  12. jet says:

    I've been using/teaching Arduino on the mac since 2009 and have had no serious problems. I download the latest IDE for my version of OS X and the appropriate FTDI driver.

    >power cycle my USB hub.

    I never use a USB hub to connect to the Arduino because there lies dragons.

  13. Check Console / your syslog. It sounds like OSX is doing its thing where it temporarily disables USB, "rate-limiting" whenever a device plugs and unplugs rapidly...

    • Nate says:

      This can be caused by current over-limits. So perhaps the hub you've got can't handle inrush current that happens at some point during programming. Does the hub have external power adapter hooked up? "Power cycle" is ambiguous if you're powering from the Mac's port alone.

      USB hubs in general skirt the limits (ignore) the spec. USB-C hubs are a special nightmare.

      • jwz says:

        The hub has external power (and a power button that resets all devices, which is what I meant by cycle). It's a USB 3 hub. Orico SuperSpeed 10, I think. I've also been using a CableMax USB Isolator though that doesn't seem to affect this problem.

        • Nate says:

          I'd still try a different hub, especially without that isolator thing. It looks like it's USB 1.1 (12 Mbps max) and that may make your FTDI serial adapter work differently.

          Or, after checking to be sure nothing is shorted out, try direct connection to your laptop. That's the best way to eliminate the hub as the problem.

  14. TreeSpeaker says:

    Are you on the latest macOSX hotness 10.12? I had, erm, USB issues cough I mean panics, disconnecting and reconnecting devices on 10.11 and 10.12 knocks silicon seems happier

  15. Dan Hugo says:

    The Arduino.cc people are pretty cool. I know them personally. If you post this question where they will see it they will probably point out some of the same things with the ftdi driver on the Mac that have been mentioned here. The community encounters these things all the time...

    Are you using branded or knock-off boards? We had some knock-offs on hand for a little project day thing and one of them burned up thanks to a bad regulator on the board. Come to think of it, what ftdi hardware is down? If it isn't real ftdi hardware i believe there is a history of poor performance (on purpose?).

    As for the IDE, well, there's always the command line and integrations with other editors and IDEs. http://playground.arduino.cc/Main/DevelopmentTools

  16. John says:

    I will echo the commenters above saying to get an old USB 2.0 hub. USB 3.0 with a hub is a crap shoot based on the machine, the hub, and the peripheral.

    I did have a problem with some Chinese arduino clones (I was drunk ebaying) instantly rebooting my Mac when I plugged them in. Not every time, of course. About 1 in 20 or 30 times, instant reboot with no warning. First time it happened, that's weird. Second time I threw them on a shelf and stopped using them.

    I will vouch for the stability of the built in USB ports, even though I am some random guy on the internet so this means nothing. My mac laptop threw up a dialog once that said "your USB ports have been temporarily disabled because a device you have plugged in is using too much power". I looked down and had shorted 5v and GND, oooops.

  17. deepatron9000 says:

    In contrast to "it just works", it's not just you. I've been to embedded hackathons were Mac users spend most of the day trying to get this working properly. Most resort to a running a Linux VM, others give up and head to the bar. Those blaming FTDI provide no evidence, since many devices use this chip (musical instruments, interfaces and such) without issue I doubt they are to blame. I'm told this wasn't alawys the steaming pile of fail on the Mac.

  18. Robert says:

    Mostly Works For Me (never use a hub though—also had the "your USB port has been shut down because the device is using too much power" on multiple occasions and send up a silent prayer of thanks to an unknown Apple hardware engineer.)

    On the Arduino IDE, oh god I feel you. I mean, I see why they did it (Write Once, It Works Terribly Everywhere!) but it's awful. But even better is that the options on the Arduino DevelopmentTools page are all worse—hey, would you like an IDE made out of JavaScript?!

  19. Chris says:

    I've had terrible trouble with FTDI drivers on OS X - not only USB giving up the ghost but also kernel panics.
    An alternative that works for me is to stick to the ATmega32U4 based Arduino devices with hardware USB in-microcontroller, such as Arduino Leonardo or Arduino Pro Micro. I find the virtual CDC serial it offers to be much more reliable on OS X.

    Another option would be to use a separate USB-serial adaptor (with a different chipset!) to communicate instead of the Arduino's built-in adaptor. Silicon Labs CP2102 is a reasonable option.

  20. lorne says:

    Like other commenters say, avoid FTDI at all costs. The CP2102 is a good choice. For the IDE side of things try platformio so you can edit in a real editor and build/flash/tail logs with their cli.

  21. PJ says:

    You could try putting a bus pirate between your Mac and the Arduino. It has a FTDI mode, so you could unplug the bus pirate from the Arduino without unplugging it from your Mac, so all the Mac sees is a stable connection to the bus pirate.

  22. Rob says:

    I've not worked with that Arduino model or MBP but I have run into the problem with Uno and Adafruit Feathers on a Mac Mini.

    One aid to troubleshooting is to set 'Show verbose output during: upload' in the Arduino IDE preferences. On upload it will output what ports it sees available (IDE v1.8.0)

    ... and then during upload, hit the reset button on the Arduino a couple of times - that sets it into a different mode (not sure exactly what - some bootloader magic...)

    That does the trick for my issues

  23. margaret says:

    i had some problems w IDE <=1.6, 1.7 100% did not work (could not see the devices), and 1.8 has been error free. programming from old macbook air, sometime through ancient hub, sometimes direct.

    also, 1.8 gives ability to program 100% through command line on linux. now you can simplify your life by programming your arduino with your raspberry pi (it's not really more expensive than a hub).

  24. Do you have more than one Arduino? Does all of them have the same problem? Do all your Arduinos use FTDI for the USB interface? Exactly what brand/model of Arduino are you using?

    I'll be glad to give you an official Arduino.cc Uno that uses an Atmel USB interface.

    I teach Arduino and although most of my users are on Macs we don't have quite so many problems. I won't say it's problem-free, entirely, but nothing like what you report and as others have pointed out more likely to have issues with Windows.

  25. Alex says:

    I wouldn't limit it to Arduino.
    I had terrible problems with a 7-port USB-3.0 powered hub on my MBP after a 10.11 point update. After working for a few weeks, the machine stopped recognizing any USB3 devices plugged into the hub after the first time post-reboot. Plugging in the hub itself filled the log with kernel messages like "AppleUSB30Hub::start: failed to set hub depth 0". Weirdly, USB<=2 worked fine through the hub, even with those errors.

    Online searches imply that the USB stack was completely rewritten for 10.11, and (some) bugs got fixed for 10.12. I eventually broke down and updated (clean reformat, actually) and the problems went away.

    Let's see if they stay gone .

  26. gryazi says:

    Only a loose data point but I've had sufficient USB bizarreness on Linux related to, apparently, either various devices or drivers that never really imagined USB devices could obtain an ID as high as USB allows - and this was mostly with USB 2.0 and I was inclined to blame the AMD/ATI chipsets involved.

    However, I'm not sure I've knocked out the entire stack that way, only fought to have my phone enumerate properly on the Nth hotplug.

    A related memory is that older (pre-thinnening, circa 2011) iMacs are extremely touchy about USB power draw, and of course the USB 3.0 Toshiba drive I got for someone's Time Machine purposes exceeded the 2.0 spec during spin-up .. sometimes. By the time I confirmed this they'd already stopped responding to recall-type requests and I eventually replaced it with a newer more power-sipping Seagate. The only reason I didn't notice keyboard loss there was that everything was Bluetooth and it usually stalled the whole boot anyway.

    It seems weird this would happen through a hub, but is your hub actually powered?

    • gryazi says:

      Noticed the power cycle step after this comment but I'm not sure if that means 'unplug hub from USB host' or 'unplug both USB and wall wart' given USB's multiple power sources - and given the number of small cheap switching power supplies I've had gradually give out in the past decade since the world gave up on big hot wasteful transformers, even the hub's power supply might be suspect too.

      I'd try to dig out an equivalent big hot wasteful transformer since hubs seem to have often used the same connectors since the 1.x days.

  27. baio says:

    Similar experiences with FTDI and OSX. My empirically based approach is to avoid FTDI USB interfaces, e.g. Arduino Micro, or use boards with no USB and use a Freetronics USB-Serial adapter which uses ATmega32U4.

  28. David Konerding says:

    The 1997 Java comment wasn't that far from the mark- the IDE came from processing, which was created in 2001. The IDE is mostly a reskin over the processing IDE.

  29. David Konerding says:

    I wish I had a formula or some magic to make your problem go away; instead, I saw your problems and just dealt with it using workarounds.

    I program a lot of Arduinos, on all three platforms (Linux, Mac OS X and Windows). All three are a bit fiddly. Macs definitely are the most fiddly- then Linux, and Windows is the least fiddly(!). Each platform has its own unique set of failure modes.

    The original Arduino Uno is the most reliable of devices; of all the Arduino-shaped things I have, that's the only one I can program very reliably. Of all the Arduinos I program, about 1 out of 10 Unos gets itself stuck in a state where I can't restore anything without programming the bootloader using an ISP.

    Many of the newer boards, like Leonardos, have a very different USB setup and programming them is far worse. ABout 1 in 5 become toast.

    Switching to the ISP (a dongle that acts as the programmer) massively improved my experience. Programming was a lot slower (30-60seconds instead of 5-10) but it worked 100% of the time on every device I had that wasn't perma-fried.

  30. fdkz says:

    Aah, FTDI with MacOS still a clusterfuck I see. 7 years ago, yanking FTDI-usbserial cable while the serial port was open in some program usually resulted in some immediate weirdness. If not immediate, then without fail, up to 12 hours later OSX just crashed out of the blue.

    Having tested cables with chips from 4 different vendors over the years, only the Silicon Labs CP2104 just works, on linux/mac/windows, even from virtual machines, baudrate up to 2Mbit, and costs less than 2$ on aliexpress. Others would crash the operating system, skip a byte here and there once per day, have limited baudrate settings, or take 20 minutes to find the drivers download page.

    So get at least 5 little CP2104 boards (they just.. go missing) for 10$ and forget FTDI even existed.

  31. Bill Paul says:

    In general, no, people don't put up with this. USB hot-swap is a pain in the neck (actually, hot-swap in general is a pain in the neck), but it's not supposed to clobber your whole system when it goes wrong. That said, in the commercial industry it's far more common to use Windows for hardware debug and bring-up. MacOS support is usually just a checkbox item. Also, they sometimes do an end-run around this problem by having a standalone debugger which you talk to from a host using TCP/IP. This also has the advantage of allowing you to debug remote systems. The disadvantage is it costs more money.

    I'm not sure how much you can blame the FTDI serial driver. In general USB RS232 drivers aren't that complicated -- they're basically just shims between the device and the serial tty stack in the kernel. The hot-swap strategy is largely defined by the OS. I suppose an individual driver could screw it up, but most of the time they're built from a template where all the hot-swap framework has already been provided and you just have to fill in the blanks for your hardware.

    I would blame Apple for its incredibly user-hostile device path naming scheme: every time you replug, you get a different randomly chosen device path name. I don't understand the rationale behind this.

    I've seen many shoops^WFTDI serial devices in my time, HOWEVER I use them mainly with FreeBSD, whose uftdi(4) driver was not written by FTDI. For that reason, I've rarely had problems with them.

    There is one irritating quirk in the OS (though it's never fatal and it's not specific to FTDI devices), which has to do with the device unit assignment. If you plug a USBRS232 dongle in, it gets assigned /dev/cuaU0. If you unplug and replug it, it typically disappears and reappears with the same name. However, if I unplug it while the terminal program (kermit) still has /dev/cuaU0 open, and then replug it, /dev/cuaU0 disappears and is replaced with /dev/cuaU1. This is due to the terminal program process still having an open file descriptor for the earlier device path: the path is removed from /dev, but the kernel can't destroy all of its resources yet because a running process is still referencing it. So instead the kernel assigns the next available unit number. Once the process exits, that path name becomes available again.

    I wonder if something similar could be happening with MacOS. (Maybe the Arduino IDE is holding the descriptor open and that's preventing a new device path from being created when you replug the device?)

    My experience with Arduino-like boards is not typical because I didn't bother with the Arduino SDK or the bootloader programming scheme. Instead I went straight for avr-gcc for development and programmed the chips directly with avrdude, initially via parallel port (like an animal) and later with a USBtiny-based programmer that I got from SparkFun. The latter does not use a custom kernel-based driver: instead you fondle it from user space using libusb. The FTDI JTAG devices (including the Olimex one John and I have for ARM devices) are also supported this way.

    Using libusb means that you're using a generic USB device driver in the kernel which comes with the OS rather than a vendor-supplied driver. I would expect the OS-included driver to be more robust in the face of unplug/replug events. It also means you don't have to go chasing down the device path every time the device reconnects, _and_ you're also not taking a detour through the serial tty stack to talk to your hardware.

    Sadly, I think your hate has to be divided among a number of people. But on the bright side, I don't expect anyone to be shortchanged.

  32. MetaRZA says:

    I hate IDEs. I hate Java. I love programming Arduinos. First thing I did was hunt down a way of compiling and uploading using a Makefile. It takes a bit of wrangling. I've only done it on Linux. I haven't used MacOS is years. A Makefile probably won't solve the problem you are having with disappearing devices. Unless the problem is that the IDE is holding the device open, but the so when you replug the OS can't use the same device name.