I want to filter some/all inbound traffic twice

[Cameron Simpson]

TL;DR: I have filtering situation that I imagine is a bit unusual
which I wish to use to work around a problem with an ISP; to do it
I need to filter some packets _before_ they're matched by the PF
states.

First the background, then what I want to do to work around the issue.

Background:

I've go an OpenBSD 5.1 Soekris box as my local firewall/NAT box.

We have an ISP connection that receives sporadic and spurious RST
packets for outbound connections. Some connections work, and some
are aborted with an RST very soon after opening.

Our ISP say that have no stateful stuff and the only real filtering
they do is packet dropping in some circumstances (I was imagining
a stateful firewall running out of state memory somewhere).

Now I have a new hypothesis; I have yet to prove it: the ISP's
backhaul has redundant links with stateful firewalls on them, and
they do not synchronise fast enough.

Under this hypothesis, our outbound SYN goes out and establishes a
connection and some state in a firewall on an outbound link
somewhere. Further packets on that connection go out and are sent
down another link whose firewall does not yet know about the new connection,
eliciting am RST from the that firewall.

This would explain why this seems to happen at the start of a
connection, but successful connections can stay up for a very long
time.

The hack:

Barring getting the ISP or their backhaul to trace this (and maybe
dispatch packets to the links based on hash instead of maybe round
robin), I was proposing to drop inbound RST packets altogether at
my local firewall.

This has obvious downsides, but should generally produce happiness
in local applications because the spurious RST packets won't abort
connections and TCP will recover as soon as the other (hypothetical)
firewalls learn the state.

To do this I need to filter inbound packets before PF's "known
states first" stuff matches the RST and passes it back in.

Because we use NAT on outbound connections I must establish state,
so I was proposing something looking like this:

ISP -> satellite-modem -> FW

Inside my local firewall:

interface-with-stateless-PF-filtering
drop inbound RST packets here

vlan (or something) with stateful rules
PF rules doing NAT etc

but I don't know how to set this up.

I was figuring the stateless rules would look a bit like this:

drop quick on $if_sat flags R/R no state
pass in on $if_sat binat to 1.2.3.4

Ignore the bad syntax on that second rule. Then internally I'd have
some kind of virtual interface with 1.2.3.4 on it running conventional
NAT rules; those currently running on my $if_sat would be moved to here.

Can someone point me in a direction to set something like this up please?
Or to suggest a better approach to prefiltering packets?

Personally I'd prefer to have a mode for PF where it ran all packets
through the rules regardless of state for some pre-filtering and
then let my put in a rule like:

pass in quick on $if_nat matching states

after the prefiltering, to do PF's usual match-states-first at that
point, with the conventional rules following.

Suggestions welcomed!

Cheers,

[Daniel Hartmeier]

If you need NAT, you have to do that on the external interface, and it
requires (implies, even) creating states.

However, you can filter statelessly on the internal interface (the
states won't match there (wrong direction, if-bound), dropping outgoing
TCP RST, passing everything else.

Sounds similar to what was done to ignore the great firewall of China,
see http://www.cl.cam.ac.uk/~rnc1/ignoring.pdf :)

HTH,
Daniel

[Cameron Simpson]

On 05Apr2013 08:45, Daniel Hartmeier <***@benzedrine.cx> wrote:
| If you need NAT, you have to do that on the external interface, and it
| requires (implies, even) creating states.

I was imagining NATing on an internal virtual interface to a private
address on some kind of internal virtual interface; this might keep
the necessary state without being the outmost layer.

And then to do stateless filtering of RSTs on the real physical
external interface (because the NAT states are not present there,
or even if floating, will not match), and then have some kind of
stateless binat or other rewrite of the remaining non-RST packets
from the real external address to the private address used for the
NAT.

| However, you can filter statelessly on the internal interface (the
| states won't match there (wrong direction, if-bound), dropping outgoing
| TCP RST, passing everything else.

Won't the RST packets shut down the TCP states as they traverse to
external interface?

I'd be happy to filter the RSTs as they exit the internal interface
as you suggest; I didn't think it would work because PF will track
the state implied by the RSTs as they enter on the external interface,
and presumably start rejecting regular traffic anyway.

Am I wrong here? (I'll try it anyway, it would be nice if it worked.)

I'd considered making the states if-bound earlier but further thought
suggested it wouldn't do me any good. But as you say, if the states
are on the external interface I would then have a free hand internally.

| Sounds similar to what was done to ignore the great firewall of China,
| see http://www.cl.cam.ac.uk/~rnc1/ignoring.pdf :)

Sounds almost identical to what they did there! Thanks for the paper;
an interesting read.

Thanks,

[Daniel Hartmeier]

Post by Cameron Simpson
I was imagining NATing on an internal virtual interface to a private
address on some kind of internal virtual interface; this might keep
the necessary state without being the outmost layer.
And then to do stateless filtering of RSTs on the real physical
external interface (because the NAT states are not present there,
or even if floating, will not match), and then have some kind of
stateless binat or other rewrite of the remaining non-RST packets
from the real external address to the private address used for the
NAT.

I guess you have already considered the obvious solution of adding a
second device (stateless filtering bridge) justs to drop the RSTs.

While it may seem a waste of parts and electricity, it's much simpler
than trying to route a connection through the same device multiple
times.

Is it a router or a bridge now?

If a router, you'd probably have to use rtables to trick the stack,
see

http://www.packetmischief.ca/2011/09/20/virtualizing-the-openbsd-routing-table/

If you have two spare physical interfaces, a patch cable between them
would be simple and produces non-surprising interaction with pf.

I don't know which (if any) virtual interface would help.

Post by Cameron Simpson
| However, you can filter statelessly on the internal interface (the
| states won't match there (wrong direction, if-bound), dropping outgoing
| TCP RST, passing everything else.
Won't the RST packets shut down the TCP states as they traverse to
external interface?

The RST packets will cause the state entries to show up in pfctl -ss
as TIME_WAIT:TIME_WAIT, but that doesn't make them not match further
packets.

The only effect is that the entries will time out (when idle!) with
tcp.closed (90s by default) instead of tcp.established (24 hours).

You can increase tcp.closed, the downside is that the entries will
stay around longer, even you'd want them to be purged.

Daniel

[Cameron Simpson]

On 05Apr2013 11:34, Daniel Hartmeier <***@benzedrine.cx> wrote:
| On Fri, Apr 05, 2013 at 07:03:52PM +1100, Cameron Simpson wrote:
| > I was imagining NATing on an internal virtual interface to a private
| > address on some kind of internal virtual interface; this might keep
| > the necessary state without being the outmost layer.
| > And then to do stateless filtering of RSTs on the real physical
| > external interface [...]
|
| I guess you have already considered the obvious solution of adding a
| second device (stateless filtering bridge) justs to drop the RSTs.

Yes, but I have failed utterly to make it work.

I have an OpenBSD 5.1 VM on my mac for trying these things and can't
even construct what I think might be needed, so clearly I do not
properly understand the vether, vlan and bridge devices (not least
because I don't understand why I get the errors I do).

I was thinking something like:

if_sat:br0:vether0

to do some stateless filtering, and:

vether1:br1:if_lan

to do the NAT etc in a conventional fashion.

However, I don't know how to interconnect vether0 and vether1 to
pass packets between them (to get the flow to be
if_sat>vether0>vether1>if_lan) so as to get two runs through PF,
and in fact I get errors trying to attach a vether to a br device.

My reading of the vlan man page suggests it is not what I want; it
seems to be a packet encapsulation, not a local virtual network.

I'll try to reproduce these errors so people can point fingers at my
conceptual errors.

| While it may seem a waste of parts and electricity, it's much simpler
| than trying to route a connection through the same device multiple
| times.
|
| Is it a router or a bridge now?

It is a router.

| If a router, you'd probably have to use rtables to trick the stack,
| see
|
| http://www.packetmischief.ca/2011/09/20/virtualizing-the-openbsd-routing-table/

I think I read this while researching a week ago; it didn't seem to address
what I though I needed. I'll give it a reread.

I gather I need to rebuild the kernel to have multiple routing
tables, yes? The installed build seems to only have table 0 (based
on errors from the "route" command when I say "-T 1" at it).

| If you have two spare physical interfaces, a patch cable between them
| would be simple and produces non-surprising interaction with pf.

Alas, I do not.

My setup is actually more complicated. The firewall has two external
interfaces: one to the satellite and one to a 3G modem. (The sat
has a bigger quota and in principle higher throughput; the 3G is
low latency).

We route DNS and ssh over the 3G (for low latency) and a few other
things (for reliability, given this RST issue I'm trying to work
around) - mostly POP, IMAP, SMTP.

Everything else goes over the satellite.

Can I use "route-to" witjout state? I think it should be ok but
haven't tried that.

We think web browsers hide a lot of the RSTs on their port 80
connections by retrying, but that SSL and other tools like iTunes
and some mail clients are pretty intolerant. Given the app errors
we see.

| > | However, you can filter statelessly on the internal interface (the
| > | states won't match there (wrong direction, if-bound), dropping outgoing
| > | TCP RST, passing everything else.
| >
| > Won't the RST packets shut down the TCP states as they traverse to
| > external interface?
|
| The RST packets will cause the state entries to show up in pfctl -ss
| as TIME_WAIT:TIME_WAIT, but that doesn't make them not match further
| packets.

So the firewall won't start sending RSTs back up the line to the
remote server? That would be bad:-)

| The only effect is that the entries will time out (when idle!) with
| tcp.closed (90s by default) instead of tcp.established (24 hours).
|
| You can increase tcp.closed, the downside is that the entries will
| stay around longer, even you'd want them to be purged.

Hmm, ok. That sounds promising. I don't really mind keeping states
around longer if I've got the memory and don't run out of ports for
NAT.

I'll put some time into your suggestion of reworking the rules to be
stateless on the LAN interface and see how that plays out. If it works
it is far simpler than making bridges and virtual networks.

Thanks,

[Henning Brauer]

Post by Cameron Simpson
| If you need NAT, you have to do that on the external interface, and it
| requires (implies, even) creating states.
I was imagining NATing on an internal virtual interface to a private
address on some kind of internal virtual interface; this might keep
the necessary state without being the outmost layer.

NAT can be applied in any direction and on any interface on recent
openbsd, so that won't stop you. the manoage has the caveats for the
respective "unnatural" direction.

you might get away with 2 routing domains.

[Stuart Henderson]

If you had spare network ports you could take the incoming feed, bridge it
to another port (filtering statelessly and if-bound), then loopback the
second port to a third port and do the normal filtering there...

I wonder if it would be possible to do similar with bridge+vether, iirc
Reyk posted a diff a while ago which allowed some kind of internal
crossconnect between devices which might be useful for such a thing.

It's all rather messy, though given what you're trying to do, a certain
degree of mess is expected ;).