We seem to have solved the "missing right channel" problem. It was, in fact, a software problem. We were running Fedora 4, and when we installed the latest patches on March 31, that's when the right channel vanished. We tried downgrading to the version of the kernel and ALSA as of three months ago, and that didn't fix it. But, Jonathan took all the sound cards home and tried them in his machine, and they all worked fine there. He was running Fedora 5. So we upgraded to that, and the problem went away.

That's right: upgrading to the latest FC4: breaks the world. Giving up on FC4 and going to FC5: un-breaks it. Nicely done, guys.

(By the way, this means it was almost three weeks before anyone mailed me to let me know that the archives were screwed up. Ten of the archives got screwed like this before I even knew to start looking into it...)

Last night was the first unattended archive after we thought we fixed it, so this morning I gave it a listen, and -- hooray! -- it's stereo again. Then I skipped forward a bit, and was almost deafened by a squealing test tone. It seems that some time around 1:08 AM, the webcast abruptly turns to a high pitched shriek and stays that way for three hours.

I'm thinking of the immortal words of Socrates who said, "OMG WTF?"

---

For years I've had it drummed into my head that you always have to keep your systems patched, if you aren't running the latest security fixes, the script kiddies will eat you alive, running a six month old OS is like leaving your front door wide open, blah blah blah. Well you know what? Fuck that noise. I'm done upgrading anything ever. The next time I get this shit into a state that seems even remotely stable, I'm never touching it again. If we get hacked, oh well. I have backups. It has got to be less work to recover from than constantly dealing with this kind of nonsense.