Just for the fun of it, here’s a a case study in troubleshooting. Actually it’s a case study in stupidity too, since I should have figured this out without all the troubleshooting. Anyway, just as an exercise…

I have an ADSL router/modem in bridge mode, with a MikroTik RB951G doing the PPPoE. The modem has a feature whereby it can still be addressed even in bridge mode. This is good, because it means you can do things like configure it, see the stats and so on. I wanted to be able to reach the modem directly from my network.

So to make the modem addressable, I put a static address on it – 192.168.222.1. I put a matching address on ether1-gateway, namely 192.168.222.2/24. The ether1-gateway interface is where the modem is connected to the router – ether1-gateway is often called the “WAN interface“.

With this in place, everything still worked. I still had Internet access and so on. The additional address didn’t seem to be confusing anything.

At this point I could ping the modem from the router and receive a response. But for some reason, I couldn’t get a response from the modem if I pinged it from my network – the network on the router’s bridge-local interface, on the “other side” of the router from the modem.

OK, I thought, let’s swing the juggernaut of scientific method into play and go through this one thing at a time.

Is the router reachable from my test host on the inside network? Yes it is. I can ping the inside address of the router, 192.169.1.1.

Is the address 192.168.222.2 on the router reachable? Yes it is. From my test host I can ping that address and get an answer. This also tells me that the default route on my test host is working. (That was a big clue right there, but I didn’t make the connection.)

Does the router know about 192.168.222.0/24? Yes it does – I can see a “connected” route in the output from “/ip route print“.

Can the router itself reach 192.168.222.1? Yes it can. I can ping that address from the router and get a response.

Are there filter rules in place on the router that might be blocking the outgoing or returning packets? No, there are not.

Are there filter rules in place on the test host that might be blocking the outgoing or returning packets? No, there are not.

OK, time to find out exactly what’s happening to those packets.

Using wireshark on the test host, I checked that the pings were exiting the test host. Yes they were – with the router MAC address in the layer 2 destination field. So far so good. But no responses were coming back.

Using “/tool sniffer” on the MikroTik, I checked whether the pings from the test host were arriving at the MikroTik:

/tool sniffer
quick ip-protocol=icmp direction=any interface=bridge-local

The pings were definitely arriving at the bridge-local interface. But no responses were leaving it, headed back to the test host.

I checked to see whether the packets were leaving the ether1-gateway interface on the router, towards the modem:

/tool sniffer
 quick ip-protocol=icmp direction=any interface=ether1-gateway

Packets were indeed exiting ether1-gateway, with the modem’s MAC address in the layer 2 destination field. But again, no responses were coming back.

So at this point I knew that the packets were definitely being delivered to the modem’s interface, but that the modem was not responding. Or, if it was responding, that something was eating the replies. Given that this was a single ethernet cable with an interface at each end, it didn’t seem likely that the responses were being lost, but it was still possible that something on the modem was blocking them.

So another test suggested itself – check for filter rules on the modem that might be blocking traffic to or from off-subnet addresses. Because ping responses happened when pinging from the router itself, I knew that pings to and from on-subnet addresses worked.

But even as that thought occurred, the magic words “off-subnet addresses” suggested another much more likely reason for all this: Perhaps the modem had no default route? Perhaps it didn’t know how to direct packets to off-subnet addresses!

And that turned out to be exactly the problem. I logged in to the modem via telnet from the router and added a default route pointing at 192.168.222.2 (the address on ether1-gateway). Lo! packets started flowing.

Update: The ADSL router/modem is a TP-Link TD-8817. It’s trickier than you might think to put a default route on this unit. So tricky, in fact, that I’ve written a blog entry about it…