the interminable Linux USB/CF saga

Dear Lazyweb, if you:

  • Use Fedora;
  • Have a reasonably fast x86 CPU (2GHz+);
  • Are able to get files off of a CompactFlash card at USB 2.0 (or FireWire) speeds;

then please tell me what motherboard and CF reader you are using so that I can go buy one.

Actually, if you're able to do it even at USB-1 speeds, tell me that too, but really, in this day and age, USB-2 should not be asking for much.

<LJ-CUT text=" --More--(17%) ">

In the last two years, I've gone through permutations of three different motherboards (and associated CPU and RAM "upgrades"), two different PCI USB boards, four different USB CF readers, switched from SCSI to IDE disks, and tried all of RH8, RH9, FC2, and FC3. To this day, the only reliable way I've found to get pictures off my camera's card has been "plug it into my girlfriend's Mac."

So as you can see, I've tried to make this problem go away by throwing copious amounts of money at it. I am prepared to continue doing so, but I don't have any fucking idea what to buy next.

Currently my motherboard is an ASUS A7V880 VIA KT880 with an Athlon 3200+ (2.2GHz.) It has onboard USB-2, and the only thing plugged into it is an IDE drive and an Nvidia e-GeForce FX 5700 Ultra.

The card readers I have tried are:

  • Vantec UGT-CR912 11-in-1 USB 2.0 reader;
  • Dazzle USB 2.0 reader, no model number apparent;
  • Lexar Jumpshot USB005 USB 1.0 reader;
  • SanDisk ImageMate SDDR-31 USB 1.0 reader.

The CF card I am currently using (though I have tried other, smaller ones in the past) is a "Lexar 2GB 40x part 2260 Rev A."

All of the card readers do some variant of this when I plug them in (card present or not):

    usb 5-1: new full speed USB device using address 52
    usb 5-1: device not accepting address 52, error -71
    usb 5-1: new full speed USB device using address 53
    usb 5-1: device not accepting address 53, error -71

    usb 5-1: new full speed USB device using uhci_hcd and address 2
    usb 5-1: device descriptor read/64, error -71
    usb 5-1: new full speed USB device using uhci_hcd and address 3
    usb 5-1: device descriptor read/64, error -71

    usb 5-1: new low speed USB device using uhci_hcd and address 2
    usb 5-1: device descriptor read/64, error -71
    usb 5-1: new low speed USB device using uhci_hcd and address 3
    usb 5-1: device descriptor read/64, error -71

Both the ehci_hcd and uhci_hcd drivers get loaded. I have kernel-2.6.10-1.741_FC3 at the moment, but I've also tried dozens of previous 2.6, 2.4, and 2.2 series kernels.

I have, at various times, seen the latter three of these cards work erratically on previous versions of Linux, though I think the last time was FC2. I have, in fact, even seen the Dazzle work at USB-2 speeds, but not lately. Generally they'd work once, then I'd have to reboot to get them to work again.

One of the Red Hat guys sent me a kernel patch that he thought might help, but it didn't. I'm so out of ideas that I offered to ship him my motherboard and all my card readers, but he declined.

So, dear Lazyweb, I turn to you.

(Don't make me ban you, ok?)

Update: Yeah, apparently I still have to spell it out for you fucktards. If you say "get a mac", I just ban you. If you say "it works great on my laptop", I just ban you. Clear yet?

Update 2: I will now jinx it by saying that I think I got it working, by 1) turning off USB on my mobo; 2) using a new "Vitex USB 2.0 PCI OHCI/EHCI" card; and 3) using the Vantec reader. (The Dazzle reader worked once, but then no longer worked after I rebooted.) We'll see how long this state of grace lasts...

Tags: , , , , ,

48 Responses:

  1. vxo says:

    Well... not sure if this'll solve the problem, but it's worth a shot...

    First of all, the only way I've had a USB removable media device play nice under Linux is if the card's inserted in the reader before connecting it. Yeah, I know it's supposed to work either way, but the kernel wants to find a partition table immediately upon device connect.

    Second, sometimes the device just doesn't fire up in time to perform the appropriate handshaking. Plugging it into a seperate powered hub may help, or if you're already using one, straight into the port.

    Third, rmmod / modprobe usb-storage. Sometimes it just doesn't wanna work right unless the driver's reloaded with it already plugged in - probably a latency/timeout issue (as above).

    Yes, USB storage is an elegant mess in Linux. I have exactly *one* USB storage device that doesn't give me all sorts of headaches... but I shall not rant about that now.

    • bodyfour says:

      There's also a new driver to replace usb-storage meant just for this situation... since usb-storage is so huge (and depends on SCSI) someone wrote a smaller "simple devices only" driver that was on lkml lately. I'm not sure if it's even in mainline though so it wouldn't be in FC3. (nor do I remember the name of it, I don't pay much attention to USB development)

      But yeah the number of usb-storage patches I see go by is pretty amazing; it sounds like there's a lot of weirdly broken devices out there that all require different workarounds to function. I guess windows and OSX manage somehow though...

      • go_team_ari says:

        It's called ub, and from my experiences with it so far, it just doesn't work very well, in addition to making things very confusing when you're trying to get usb-storage but end up using ub instead (since hotplug seems to prefer ub over usb-storage).

        • terras says:

          Specifically, if ub is enabled, certain USB device IDs are ignored by the usb-storage driver. This has caused a riduculous amount of grief, since RedHat, in their in infinite wisdom, enables everything as a module for their kernel builds. They may have gained a clue and disabled ub by now.

          • cowbutt says:

            $ grep UB /usr/src/redhat/SOURCES/kernel-2.6.10-i686.config
            CONFIG_I2C_STUB=m
            CONFIG_USB_EHCI_ROOT_HUB_TT=y
            # CONFIG_BLK_DEV_UB is not set
            CONFIG_MINIX_SUBPARTITION=y
            CONFIG_CRYPTO_ANUBIS=m

        • Isn't it still marked as something like "experimental low speed USB driver"?? That put me right off.

    • edolnx says:

      You can get around the "plug media in device first" problem by running rescan-scsi-bus.sh as root. It's not a default utility - but Google can find it. It will force userspace to reprobe the kernel and bring out all the new SCSI (and thus USB Storage) devices and make their /dev entries work. It's the only way to even get the /dev entries to show up if you are using udev or devfs (at leat reliably).

  2. ioerror says:

    Manufacturer: Belkin Mfg Part #: F5U140

    This works under Debian with a stock 2.6.8 kernel from kernel.org. I assume it should work just fine with your distro, if you want to try before you buy, let me know. However it's USB s1.0W

    • In that vein this Sandisk unit has worked for me and my friends on every Linux, FreeBSD, OpenBSD, OS X, and Windows system we've tried it on without any issues or recompiling. It's USB 2.0 but can do 1.1 if you need. I love the thing for more than just it's working behavior because it comes with a stand for the desk and a short cable for travel.

      I know you've tried SanDisk already, but the newer SanDisk products seem to be doing a fine job.

  3. baconmonkey says:

    now it may seem strange to go with the obvious solution of "go with what works", as opposed to "bash brains on wall trying to make it work"... but why not spend the money on a mac mini and a KVM, which you know will work, as opposed to inflicting upon yourself the JWZ kryptonite of Rebuilding The Kernel?

    • sc00ter says:

      Just because it's a mac does not make any USB device automatically work. I had a CF/SM card reader that was USB. Worked like a champ in Windows. Under both linux and OS X the CF side would work, but it would not see the SM side of the device.

      Of course my digital camera is SM, I got the dual card because I saw using CF in the near future. I eventually got another one that does work, I forgot what kind and it's at home right now.

      But just saying "go with a mac" may not solve all his problems, now, and int he future.

      • c0nsumer says:

        Mind me asking which reader you had? I had a slightly older Memorex USB 1.1 CF reader which worked great under Windows and Linux, but it just didn't work on my Mac. It has since been replaced by a Dazzle USB 2.0 multi-format reader and a PCMCIA adapter which seems to work great on everything I've tried (FreeBSD, OS X, Knoppix, and Windows).

      • baconmonkey says:

        To this day, the only reliable way I've found to get pictures off my camera's card has been "plug it into my girlfriend's Mac."

        also, I hate Macs. I'm not going to enter a holy war on that subject. I know JWZ won't go with windows, which is pretty well guaranteed to work with a USB card reader, since 99% of them are marketed towards a primarily windows audience, so a Mac would be the next best thing, especially since he's already stated that they work for him.
        We also all know how much JWZ loves him some kernel recompiling.

    • cetan says:

      Or at least buy the girlfriend a new mac and take hers because you know it works :)

    • fantasygoat says:

      I was thinking the same thing but didn't want to risk the banning.

      OS X is making a convert of me - all my usual command line UNIX fun and Shit Just Works™.

      But I certainly understand his stance of wanting to stick with Linux - all this stuff should work and there's no real reason why it doesn't. He should be able to use what he wants the way he wants.

    • jwz says:

      You just don't fucking listen, do you?

      *PLONK*

  4. remaker says:

    http://www.linux-usb.org/FAQ.html#ts6 ?

    Looks like a mobo/BIOS issue, or a PCI shared interrupt issue ?

    You thought the state of Linux Video was bad, I can hardly wait to see the gruntle on USB mass storage under Linux. I have recently gone through a saga of trying to make a number of USB sticks into bootable devices (in Windows), and found that the quality of implementation (in hardware) of USB mass storage devices is all over the map.

    One google suggestion says:

    You may be able to avoid this problem by loading the usbcore.ko module
    with module parameter:
    old_scheme_first=y

    And there is a lot of stuff including developer rants at

    http://www2.one-eyed-alien.net/~mdharm/linux-usb/

    I suppose it should tell you something if the main developer of USB mass storage for Linux is a one-eyed alien.

  5. wfaulk says:

    Both the ehci_hcd and uhci_hcd drivers get loaded.

    I had a similar problem a while back — transfer speeds being way slow — and I seem to remember those two modules getting in each other's way. Currently I'm just loading ehci_hcd and everything works fine, including speedy USB storage transfers. I'll admit that it's not from a CF card, but a USB2 hard drive, though.

  6. Apologies in advance for being a dumbass and posting something that I haven't tried, but have you considered a straight-up PCMCIA ATA IDE interface? (God, so many acryonms!) A minute or two of googling turned up this promising-looking item from Addonics. It's got a PCMCIA slot on the front, and an IDE cable on the back. No drivers. It seems pretty clear that you're using a desktop, so the fact that it's internal and not USB may not be a problem for you. The product page claims that it requires Windows, but I'm not sure I believe it. The data rate of 16meg/sec exceeds the speed of most (all?) CF cards. Anyway, it's like $40, so you might consider it.

    • Also this, which claims to work with Linux.

      (If y'all don't know, CF disks are almost exactly IDE drives, so it's pretty trivial to hook one up to an IDE cable and let it rip.)

      • phs says:

        <anal-retentive>
        Technically, the ATA-lookalike mode is only one of three different modes that are selected on powerup of the card. When used in a PC Card slot for instance it uses a mode in which the pins look much like an ISA bus IDE controller (which is slower than hell, which is why a lot of people are starting to use them in the "True-IDE" mode with another controller.)

        Also, newer CF cards are starting to support UltraDMA (by stealing two lines for DMARQ and DMACK) when used with an ATA controller, which makes them even faster.
        <anal-retentive>

        Unfortunately this probably isn't going to help Jamie with his problem - he'll be trading USB problems for IDE hotswap problems.

        • treptoplax says:

          There exist floppy-disk memory card adaptors (literally, you put the memory card into this thing and stick it in a 3.5" floppy drive).

          Unfortunately, I've only ever seen/heard of those for Sony Memory Stick and 'SmartMedia', not for regular CF (is SmartMedia a superset of CF? I don't even know). If such an animal exists it would probably work, though very very slowly.

          • bdu says:

            Back when I was using this method on an old Sony, not only was it dog slow but it required a special driver to see the data from the stick inside the floppy adapter, so it would likely be a no-go on two accounts (too slow and bad on linux).

    • jwz says:

      Three years ago, I tried to get a PCMCIA bay working in a desktop machine (two different card models.) It was a total clusterfuck. As far as I can tell, the machine spun out directly into IRQ-conflict-hell. I spent nearly six months trying to get them working before throwing them away.

      My conclusion: "PCMCIA only works on laptops, because only laptop hardware contains whatever the necessary special-cases are."

  7. octal says:

    I generally use a pc card to ata direct interface, either on laptops, or as a 3.5 bay device with cards; much more reliable than any usb based solution I've ever seen.

  8. jkonrath says:

    You could always skip the middleman and get one of those all-in-one reader/burner things to dump your cards directly to CD, and then read those at your leisure on any computer. There are models with battery packs that are pretty small, so you could easily burn a CD in the field if you're taking a lot of pictures. The only limiting factor is that they are usually CD-only; it would be better if they made one with DVD to empty out 2 Gig cards.

  9. krick says:

    I just wanted to back up <lj user="vxo">'s comments regarding having a card in the USB reader before connecting it.

    My girlfriend has the SanDisk ImageMate SDDR-91 unit.

    She's running windows but the problem seems to be the same.

    If you don't put a card in before plugging it it, strange things happen. In addition, the OS sees it as a removable drive or something when the card is removed but the reader is left plugged in.

    This new removable drive then somehow confuses software protection like that in "The Sims". It complains that you don't have the CD in the CD-ROM drive when you do.

    I guess it thinks the card reader is the CD-ROM device or something.

    The only solution we've found is to leave the reader always plugged in and keep the 16M card that came with her camera in the drive all the time unless we need to download pictures. In which case we swap cards.

    If I were you, I'd try an IDE flash adapters or a PCI flash adapter. I think USB is just too flaky for this.

    • Count yourself lucky. My roommate's reader actually reboots his Windows XP machine if he hotplugs in. So he (like you) just leaves it plugged in. Showed up fine on my old Mandrake desktop, though.

  10. bitwise says:

    Note also that it's not necessarily the reader that doesn't work with Linux, but the card itself. I know that when I went out and bought a bigger CF card I couldn't read it at all. The old, smaller card works fine.

    Sadly, I think this is more common with newer, faster CF cards (i.e. Lexar).

  11. edolnx says:

    I get the same problem with any USB device on my system (a VIA KT400 based mobo) once I've gone through 6 USB plug events. After much searching and testing, I think my problem (and yours) is that the VIA USB hardware stack sucks. My suggestion is to grab an OHCI based USB controller (chipset made by anyone other the VIA or Intel, usually found on the cheaper cards), disable all USB on your MoBo, and give that a whirl. I'm going to do the same on my system and see if it helps. The good news is that a 2-4 port OHCI controller is around $20 at Fry's, and that most of the "no-name" cards are OHCI. I'll follow up tonight if the new card works.

  12. dachte says:

    My system works fine with an el-cheapo USB1 reader called the FPTS-C (see http://www.wirelessworld.com.au/FPTS-C.htm ). My system is a laptop -- an Inspiron 8500. The usb controller is, according to lspci,

    00:1d.7 USB Controller: Intel Corp. 82801DB (ICH4) USB2 EHCI Controller (rev 03)

    Hope this is useful.

  13. splashed says:

    > To this day, the only reliable way I've found to get pictures off my camera's card has
    > been "plug it into my girlfriend's Mac."

    > So as you can see, I've tried to make this problem go away by throwing copious
    > amounts of money at it. I am prepared to continue doing so, but I don't have any
    > fucking idea what to buy next.

    I think you've answered your own question. http://www.apple.com/macmini/

  14. lazy_web says:

    Dear jwz,

    I've had it up to HERE with your goddamn whiny questions. I don't care about your shitty computers in your shitty club. Figure it out yourself and STOP BUGGING ME, YOU FUCKING MORON!

    Love,
    Lazyweb

    P.S. What kind of idiot expects tech advice from his Livejournal anyway?

  15. jonabbey says:

    I'm using a Dazzle CF reader under FC3. I don't know from USB 2.0, but it just works with the proper entry in /etc/fstab:

    /dev/sdc1 /mnt/flash auto noauto,owner

    I've never had any issues, so I don't know what else to say about that.

    I'm running on an Intel motherboard, the 865PERL one.

  16. reify says:

    I've got USB1 devices working here, and they're running off USB2 hubs, which seem to be recognized by the OS.

    OS: Fedora Core 3; kernel 2.6.10-1.741_FC3
    CPU: Intel(R) Pentium(R) 4 CPU 2.40GHz
    Motherboard: Gigabyte GA-8PE667 Ultra

    Using:
    Dazzle USB reader, no model number.
    integrated cardreader on HP 7960 printer

    Both these devices are connected through a Belkin USB2 hub to one of the motherboard's USB ports.

    Here's what I get:

    Feb 1 19:57:00 localhost kernel: usb 2-2.1: new full speed USB device using ehci_hcd and address 6
    Feb 1 19:57:00 localhost kernel: scsi1 : SCSI emulation for USB Mass Storage devices
    Feb 1 19:57:05 localhost kernel: Vendor: eUSB Model: Compact Flash Rev: 5.09
    Feb 1 19:57:05 localhost kernel: Type: Direct-Access ANSI SCSI revision: 02
    Feb 1 19:57:05 localhost kernel: Attached scsi removable disk sdb at scsi1, channel 0, id 0, lun 0
    Feb 1 19:57:06 localhost scsi.agent[6252]: disk at /devices/pci0000:00/0000:00:1e.0/0000:02:07.2/usb2/2-2/2-2.1/2-2.1:1.0/host1/target1:0:0/1:0:0:0Feb 1 19:57:06 localhost kernel: Device not ready. Make sure there is a disc in the drive.
    Feb 1 19:57:06 localhost kernel: Device not ready. Make sure there is a disc in the drive.
    Feb 1 19:58:05 localhost kernel: SCSI device sdb: 1000944 512-byte hdwr sectors (512 MB)
    Feb 1 19:58:05 localhost kernel: sdb: Write Protect is off
    Feb 1 19:58:05 localhost kernel: sdb: assuming drive cache: write through
    Feb 1 19:58:05 localhost kernel: SCSI device sdb: 1000944 512-byte hdwr sectors (512 MB)
    Feb 1 19:58:05 localhost kernel: sdb: Write Protect is off
    Feb 1 19:58:05 localhost kernel: sdb: assuming drive cache: write through
    Feb 1 19:58:05 localhost kernel: sdb: sdb1
    Feb 1 19:58:06 localhost fstab-sync[6300]: added mount point /media/TOSHIBA512M for /dev/sdb1
    Feb 1 19:58:06 localhost kernel: FAT: utf8 is not a recommended IO charset for FAT filesystems, filesystem will be case sensitive!
    Feb 1 19:58:06 localhost kernel: SELinux: initialized (dev sdb1, type vfat), uses genfs_contexts

    "full speed" = USB1, and it runs about that fast:
    [root@spiffy ~]# ls -l testfile
    -rw-r--r-- 1 root root 8388608 Feb 1 20:07 testfile
    [root@spiffy ~]# time cp testfile /media/FLASH_128/misc/testfile

    real 0m19.984s
    user 0m0.003s
    sys 0m0.119s
    [root@spiffy ~]#

    Printer works similarly, but is slower.

    Didn't do any special configuration. This setup also worked under Knoppix (used the flash reader to copy my bootsector over to Windows for dual-boot).

    Hope that's helpful.

  17. violentbloom says:

    You know I think you *like* the pain.
    isn't the using your girlfriend's mac solution really a lot easier? also free! plus if you ignore this problem long enough there will be some other new technology to use instead, though it probably still won't work with linux.

    • jwz says:

      No, it's not a lot easier because then I have to copy these gigantic files over the net, which takes fucking forever.

  18. mjg59 says:

    Do any USB devices work on this system? The symptoms you're describing are fairly typical of the ones you get where the USB interrupts aren't routed correctly (there seems to be some alternative non-IRQ-requiring way to get notified about device insertions, but then nothing else ever works). What does /proc/interrupts look like? Do the USB interrupts increase when you plug/unplug stuff?

    • krick says:

      The frickin lovely thing about modern motherboards is there's lots of onboard stuff now and still only 4 IRQs to go around.

      This is what the ASUS P4P800 Deluxe motherboard has the following items riding the PCI bus...

      video card (AGP shares the PCI bus)
      onboard sound
      onboard NIC
      onboard 1394 (aka firewire)
      onboard RAID controller
      USB controller #1
      USB controller #2
      USB controller #3
      USB controller #4

      ...and there is only 4 IRQs shared between all these devices!!!

      The USB controllers HAVE to share IRQs with other devices.

      It's a wonder anything works.

      PCI devices are *supposed* to share IRQs but I still don't think they've got all the bugs worked out yet. Just try having your Soundblaster Live sharing an IRQ with your SATA RAID controller to see what I mean.

  19. cowbutt says:

    I've used a number of usb-storage devices - a Lexar Jumpshot cable and a 16MB CF card, an Olympus C-750UZ digicam and a handful of Prolific PL-2507/PL-3507-based ATA->USB caddies (the 3507's firewire is pretty awful, but that's another post).

    I've generally always used said devices with Intel-chipset motherboards; an Asus P2B (i440BX based) and a couple of Gigabyte GA-8PE667 Ultra's (i845PE with built in USB + second, slower, NEC USB 2.0 controller). I've seen posts describing various problems with VIA chipsets and USB, so that might be your problem.

  20. chnrxn says:

    I have the same problem as you, but I was using one of the latest 2.4.30 kernels on a Redhat 9 installation, which apparently still has very old USB drivers.

    I popped in a Knoppox 4.0.2 LiveCD running a 2.6.12 kernel and got my thumb drive running at USB2.0 speeds with the EHCI-HCD driver.

    Being too lazy to compile my own 2.6 kernel, I copied the kernel and the /lib/modules/ directory from the Knoppix CD (while Knoppix is running), and after a few adjustments, am running off this kernel successfully.