Sys y u no log?

I can't get my DSL modem to syslog to my iMac with an Airport inbetween.

I've got: DSL Modem → Airport → iMac.

DSL is Zyxel P-663HN-51 ADSL2, software 1.01(BOM.3)b1_0824. Airport Extreme is 7.6.1.

The Zyxel is in bridge mode. The Airport runs DHCP/NAT, has IP 70.whatever, LAN 10.0.1.1, and default host 10.0.1.2 (the iMac).

Airport is correctly sending syslog to the iMac, but the Zyxel is not. So the problem's not with the Mac.

I tried setting Zyxel's log host to both 70.whatever and to 10.0.1.2 and neither work.

Does the Airport not route UDP, or what?

Of course the only reason I'm trying to do this in the first place is to figure out why, about once a week when a few big, well-seeded torrents come in at once, the Zyxel loses is mind and has to be power-cycled.

I've even pulled Transmission's global connection limit all the way down to 20. Still happens.

Everything is terrible.

Tags: , , ,

34 Responses:

  1. Tim says:

    Try adding a specific port forwarding entry in the "Port Settings" area and also look at this bug. That may work where the "default host" routing does not.

  2. Might be worth dropping a hub (they still make those, right?) in between the zyxel and the airport, and then firing up tcpdump to see if the zyxel is even emitting syslog packets in the airport's direction. Airports will most certainly route UDP packets, so I doubt the problem is there. But some random embedded system's syslog server? God knows.

    (I suppose you could also try setting up a static port binding for 514/udp, but the 'default host' function should certainly cover that.)

  3. dinatural says:

    I don't think the Airport will make anything coming from outside to inside unless specifically set (or dynamically generated by an inside connection waiting for outside stuff). And I had troubles like that with a somewhat "old" version of transmission, maybe check for that...

    • jwz says:

      Added explicit rule, no change. Latest Transmission.

      This would also be much less of a pain in the ass if I had a way to access the Zyxel config that didn't involve changing my iMac's IP address and physically moving the ethernet cable. Sigh.

      • sherm says:

        What IP do you have to give the Mac to talk to the crappy thing when it's in bridge mode? Can you plug some other thing into it, put it on that subnet, and try to syslog to that?

      • Tim Bradshaw says:

        Summary: I have what I think is a very similar configuration, have tried the same things, and I am convinced that this is not possible. I'd like to be wrong about this.

        I'm using a Time Capsule (which I kind of hope is the same as an Airport in terms of IP) and an ADSL modem (different model) in bridge mode on the far side of it, with the Airport talking PPPoE. I've been through the same stupid having-to-reconnect-things to talk to the modem and wanted to get syslog from it.

        Here's why I think it can't be done. The Airport is talking PPP on its WAN interface, which means that the WAN IP address essentially lives somewhere at the far end of the PPP connection, in the exchange or the ISP's network somewhere. There is no IP-level network between these two points, there's just ethernet between the Airport and the modem and then ATM (or whatever it is, I think it depends on DSL flavour) on the far side of the modem, with PPP over both these.

        Opening up a port on the Airport opens it from the public IP on the ISP's network into your internal network, but the modem can't see the public IP. A test of this would be to send some syslog data from a machine outside your network to your public IP, and it should arrive at your local syslogd.

        If the Airport was a proper thing, then you could configure an address on the WAN interface (in addition to the PPP tunnel) and then set up a little subnet on that bit of ethernet with routing to your internal network. But as best I can tell there is no way you can do that from Airport Utility, so at that point I gave up.

        One thing I thought of but did not try: there isn't any real reason why you could not have another device on the ethernet on the WAN side of the airport, which could then route from that subnet to inside. In fact that little network does not even need to be a different subnet, so you could just use a switch. The problem with doing this is that the PPP traffic needs to go to the modem, and if you put a switch in the way it won't, it will go to the switch. I think it would work to get a switch on which you can set up port mirroring, with the Airport's WAN port and the modem's port set up as mirrored. May be a hub would work as well, if you can get hubs still.

  4. Nick Lamb says:

    What's with all this private IP bullshit? Presumably the Airport Extreme is doing address translation? I can't swear that's making things worse but it definitely makes things harder to diagnose.

    • jwz says:

      Because ISPs don't exactly hand out Class Cs any more, grampa.

      • Vincent Janelle says:

        I think he was implying that if it's bridge mode, it should be acting as a layer 2 device and only forwarding multiplexed ethernet frames from the ADSL side? Ie, anything on the lan side of your zyxel should be the internet, not some double-nat mystery(which could be the cause of a state table filling up, or connections never getting reaped). That is, unless your ISP is doing carrier-grade NAT and you'd get a RFC1918 address from them.

        Is the Zyxel sending syslog out TCP/514, instead of UDP/514? Some syslog implementations will do this.

        Can you change the destination port? The airport might be rejecting connections to that.

        • jwz says:

          There's no double-NAT. The Zyxel is a bridge, the Airport does NAT. So Zyxel should be able to get to iMac via the 70.whatever address, I guess. The OSX syslogd doesn't seem to listen on TCP.

          • Vincent Janelle says:

            Ah, so zyxel -> x.y.z.a rfc1918 <- imac?

            I'd hook the imac to the zyxel and `tcpdump port 514`, login and see if it produces some traffic. I'd be really surprised if the airport was actually listening to 514 on that device, and even then the internal forwarding should still proxy it.. Normally these devices implement the lan/wan as seperate vlans instead of discrete L2 devices..

            I'd be really shocked if it was doing TCP/514 however.

            Is the zyxel device 'yours', and is there a firmware update for it?

            WRT to the crashing, I had a WRT54GL do much the same - it was due to some nat tables having excessively long periods in between expirations, and the device would simply just run out of memory if I ran bittorrent, due to sheer number of hosts you connect to for a long running torrent. I'm not familiar with your device however.

            • Vincent Janelle says:

              Neat, your blog chewed up some of the formatting.. there's a missing airport in between x.y.z.a and rfc1918.

      • Nick Lamb says:

        Actually real ISPs never stopped handing out address blocks. Even in regions where the RIR has entered its v4 exhaustion regime, the LIRs were never so much as asked to start doing this Mickey Mouse private network bullshit to their customers. And ARIN isn't in exhaustion, the LIRs where you are could get more blocks any time, they just choose not to because it represents an infinitesimal cost savng for them, and the only person being inconvenienced is you the customer so why should they care if you won't make them?

        I have a /28 here since I got signed up with my current ISP literally last month, and that's in RIPE where there actually is exhaustion. The idea that ISPs are no longer able to hand out address space is a convenient fiction, you'd never hear them say that when talking to the RIR, only to customers who they think don't know any better.

        • jwz says:

          Your rant is fascinatingly irrelevant. Knock it off.

          • Different Jamie says:

            That was pretty classic, no?

            "I have two devices that fail in this mode."

            "Let me tell you how the Internet Really Works, from the perspective of a retail buyer."

            I don't have much useful to add, other than my Airport Extreme randomly crashes. There is an Express that exists mainly to support some speakers, and a Belkin something or other on the other side, because brick wall in that end of my home, but it craps out maybe every other week. Personally, next time around I'm likely to get one of the industrial Motorolas. In SOMA, there is so much contention that I'm surprised it works for anyone.

  5. Logan Bowers says:

    If you're doing RFC1483 bridging and the Airport is handling authentication, then your Airport is decoding PPP packets wrapped in ethernet frames, and will not see any normal IP packets coming into its WAN interface. The Zyxel is certainly not smart enough to wrap up its logging packets in the right goo to transit a router. You would need to place a host on the same segment as the Airport and DSL modem in order to see the syslog packets.

  6. wkrick says:

    The number one reason that causes network hardware to "lose its mind" is improper cooling. They never seem to design things with enough heatsinks in the first place, then they make the problem worse by designing the cases for appearance over functionality, so they have shitty ventilation.

    • Grey Hodge says:

      Considering it's in bridge mode, this is also my guess. Stick it next to a fan and see if that helps. It's consumer level hardware and you're throwing more work at it than it's used to. It heats up, bits flip and it starts talking to hyperspace.

  7. The Zyxel only has 192.168.0/24 configured, thus it will never know how to route to either the IP address the Airport gets from PPoE and it's 10.0.1/24 network. The Airport doesn't retain routes to the 192.168.0/24 and instead does 0.0.0.0/0 to the PPoE endpoint.

    As an aside, the Zyxel is a massive piece of shit. Back when I had AT&T DSL, my Zyxel would fold under heavy load (and especially torrents) like a two dollar Chinese made Walmart lawn chair when I flop my 330lb ass down on it.

    I would get a running start and chuck that plastic turd as far as it will go and find any other modem. Here is a list of what AT&T supports, buying on eBay, of course.

    • Ben says:

      This assumes the Airport is talking to the ZyXEL over PPPoE rather than IP. You can tell if that's the case because if it is you put your ISP password into the Airport rather than the ZyXEL. If this what is happening then what you want to do is
      - tell the ZyXEL to use 10.0.2.1/24 on its internal interface
      - tell the Airport to use 10.0.2.2/24 on the interface connected to the ZyXEL (this doesn't (or shouldn't) interfere with the use of that interface for PPPoE)
      - tell the ZyXEL to route 10.0.1/24 via 10.0.2.2
      - tell the ZyXEL to log to 10.0.1.2
      and both syslog and telnet/http should start flowing between the iMac and the ZyXEL.

      I don't know if the Airport configuration supports this. With the ZyXEL box I used to use (a P-660HW-61, but I expect the commands are the same) routing could be configured with
      ip route status
      ip route add dest/mask gateway
      ip route lookup addr
      from the command interface (found under 'System Maintenance' on the telnet interface); I don't remember if you could configure this using the Web interface.

  8. Andrew Stern says:

    Your real problem is a DSL modem on the fritz. It should be replaced.

    You are seeing problems with the modem DSP chip burning out OR the power supply going out.
    This happens -all- -the- -time-.

    Go to Central or Fry's, and purchase any cheap (tp-link or whatever) adsl2+ modem. Or get at&t to overnight you a 2-wire box. Any other course of action is a waste of your time.

    This is not an IP problem, as your modem is in bridge mode and not handling the packets in any meaningful way.

    • gryazi says:

      Actually, this sure as hell is a problem with 2Wires where the switching wall-warts seem almost purpose-built to gradually crap out with curious and horrible intermittent-gradually-increasing results (did you know crypto with a long WPA key seems to draw significantly more CPU current than crypto with a simpler key?).

      I would expect it to be somewhat less of a problem with other hardware, but it could be a different shit-tastrophe where the modem decides to fail to retrain or downgrade when it actually has to, which may become more apparent as components age and maaaaybe fewer chunks of DSL-spectrum are actually cleanly usable.

      "Blindly replace hardware" is suboptimal but "borrow a working one or just grab a returnable Netgear at Staples to prove you're not crazy" should probably go on the list after the first straight 24 hours lost to fucking with it.

      [And there's nothing so fun as paying for a telco service call just to prove that your bridge-only Speedstream's input circuit is fried and yes you really just need to just replace the modem and since it's not AT&T's fault that'll be $$$ please, but that was a consistent degradation to sub-dialup speeds rather than a load issue so meh.]

    • jwz says:

      This is not hard to believe, but I sure wish I could get fucking syslog working on the off chance that there would be something in there to indicate this.

      • gryazi says:

        Is the ZyXel actually so sadistic that it claims it can syslog but has no web-interface log buffer-viewer? Or is that coming up empty after a freeze so you're just hoping to catch it in the act?

        (This is just morbid curiosity. 2Wire deathspirals can usually be diagnosed by watching it reboot, swapping in a PSP wall wart - same 5V 1A-ish and connector - then canning it when it continues to log the mysterious "DMA_ERR" and connection drops/resyncs after trying every combination of resets and 'only use this combination of settings' red herring suggestions. Entirely different issue just redocumented here for the lazyweb, though.)

  9. jml says:

    I'm more familiar with cable than DSL, but have you ever actually proven this is possible at all? Could your U/S provider be blocking syslog out that interface?

  10. Phil says:

    I have next to me a Speedstream 4100 and power supply that I will mail to you for the cost of postage. at least you can do an A/B test. I'll even put it in bridge mode before I send it to you.

    I would agree with the peanut gallery on the diagnosis of "hardware problem." Few software problems cause a hard I-need-to-power-cycle reboot. And even if it was a software problem, good luck getting a firmware update.

    For less than $20, you can get a DSL modem from a fellow city-dwelling resident that switched to U-Verse or Comcast.

  11. Zygo says:

    Is the ZyXEL doing some kind of packet inspection? I've seen "bridge" devices that track connection state anyway, because their vendors get lazy about firmware and the "NAT off" switch just means "turn off the web UI" instead of "cease using scarce resources to implement undesired non-mandatory features that might cause starvation." Busted firewall code pops up in surprising places, like "dumb" Ethernet switches that aren't supposed to have user-visible firmware at all, but they have it because the much-more-expensive "smart" managed switch made by the same company runs on virtually identical hardware, and they failed to hide it completely. These devices will crash if there's a complex TCP topology passing through them. They'll also do things like reply to any ping they see with duplicate pong packets from phantom IP addresses, but only if there was no DHCP server on the LAN when they powered up.

    Big torrents (with 10k+ peers) crush many embedded NAT implementations like a bug. All of the peers with unsaturated uplinks want to connect to you, and the cheap NAT craps out before even 1000 connection states go by, let alone ten or twenty times that. "craps out" ranges from "starts dropping established connections at random" to "you have to power cycle the device because in a competitive market not everyone gets to do adequate QA before they ship."

    Transmission settings don't count here--these problems, if they are what you are having, arise from incoming connections, over which you have no control (although refusing them at the firewall might help a little).

    If you don't care about having a firewall at the AirPort, you could connect both sides of the AirPort and Zyxel to the same Ethernet switch (not hub). The Zyxel won't listen to random IP traffic going by, and everything else won't speak PPPoE to the Zyxel, and the AirPort will give different MAC addresses for its different Ethernet interfaces, so all the traffic will end up in the right places as long as nothing deliberately tries to communicate with something it's not supposed to. Then you just give the ZyXEL your iMac's private IP address to use as a log host. Any Ethernet switch will do--possibly including one in the AirPort itself if there are enough ports.

    If your time has non-trivial monetary value, buy a different cheap DSL modem every week, drop it into your existing setup, and stop when you have one that is no longer crashing under your specific workload. Give the extras to your less-network-intensive friends if you can't give them back to the retailer. Buggy DSL modems can still be useful for someone, even if they're useless for you.

  12. err0neous says:

    If the Zyxel has no IP address then I don't think it can send its own IP packets (e.g. syslog). If I'm wrong and someone knows why, I'd be interested to know the reason.

    • jwz says:

      There is a web interface log viewer, but I can only get to the web interface by changing my IP to 192.whatever and plugging in directly instead of via Airport. So it's a monumental pain in the ass.

      • gryazi says:

        Didn't bother phone-typing this earlier, but my suboptimal it's-the-future solution has been to stockpile 'disposable laptops' at the various places I expect to possibly be stuck in this sort of hell (and being laptops, they tend to be draggable-to-source-of-problem except that you will never have carried one when the problem arises).

        Since it is The Future, the level of hardware necessary to do basic troubleshooting and poking-at-web-interfaces has now sunk to... $free. [Even Intel Macbooks have been around so long that I think I just saw some going for barely-three-digits in surplus, although OS X is not my first choice for reliably troubleshooting network BS given how reticent it tends to be about why it thinks an interface needs to be down/Safari deciding to be adamant that you have no Internet connection when you do but it's an unreliable link etc.]

        I was kinda not going to bother thinking this out loud until I realized you're probably surrounded by people who have fully-loaded netbooks collecting dust because the whole netbook craze lasted what, 8 months half a decade ago now? I can tell you don't want another piece of crap on your desk, but balancing it there temporarily and then stuffing it in the drawer until the next 'emergency' beats the whole cable gymnastics routine.

        [As a Ubuntard who as-often expects to have to poke at something in OO.o/Libre, note that if you figure "screw it, Ubuntu is almost like having a desktop OS" you need to write off the whole first generation of Pentium M hardware to not start out end-of-lifed because they decided W^X is a must-have but soft emulation of it is no longer worth maintaining. But somewhere in your neighborhood someone is already throwing 64-bit or dual-core hardware straight into a dumpster because it's 'old crap'. Any of the BSDs will probably run on anything you could find without complaint and getting to twm + Firefox is only like an 'install 2 packages' pain-free 5 minutes here in 2013.]

        I try to just think of them as IP-vt100s with a lot more vulnerabilities.

  13. Malcolm says:

    Run a second ethernet cable from a free port on the Zyxel to a free (LAN-side) port on the Airport. Config the Zyxel so the IP of its web interface is 10.0.1.*

    Srsly.

    • jwz says:

      Oh, hey! That trick works. Well, it works for letting me get at the Zyxel's admin page without having to move cables around. I'm still not getting syslog, though, even after setting the Zyxel's log host to 10.0.1.2 (my iMac on the LAN side). Well, at least now I can see it through the web interface without taking everything down.