[Novalug] "taint"

Rich Kulawiec rsk@gsp.org
Sat Mar 28 10:40:57 EDT 2015


On Tue, Mar 24, 2015 at 09:39:37PM -0400, Peter Larsen via Novalug wrote:
> Since you seem to just be talking about URLs, you're not covering the
> most common use case with POST forms. Most data submissions aren't done
> through the URI but through http parameters.

Yes, yes, yes, I *know*.  I deliberately used simple examples in order to
illustrate the point.  Do recall that we started talking about validating
input in the conext of Perl's "tainted" variables; I'm extending that
to the more general practice of never trusting input.

That includes input that your own operation generates (as output from
something else).  Trust no one.  Validate everything.  In other words,
if your web server is emitting DB queries and directing those to a DB
server, do not blithely trust that all those queries are well-formed.

> All you've done is push what the web-server is doing to a different
> server. So when you have changes, you now have to change both the proxy
> and the real server, and hope you cover all functions properly on both
> sides? Not sure I see the safety here.

Well, changing multiple things in synchrony is nearly always a trivial
problem, and even when it's not, tools like make, rsync, perl (or if you
like, chef, puppet, ansible) make it easy.  So that's a non-factor.

And the security point *is* that it's a different server.  This is key
for several reasons: first, it adheres to the fundamental principle
that abuse/security measures should always be deployed as close to
the source/as far from the target is possible.  (Example: you *could*
block spam from DROP-listed networks in your MTA on your mail server.
But a far superior solution is to null-route those networks in your
perimeter devices, so that they never get anywhere near your MTA.)

Second, HTTP or FTP or SQL servers, as we are painfully aware, are
not bug-free.  Neither are operating systems.  Neither are [fill in
the blank].  Interposing a second server running a different HTTP
server/proxy means that an attacker attempting to exploit a bug that
neither you nor the software author know about has to figure out
how to get by both.  This raises the bar considerably.  (As does
using a different operating system and/or different CPU architecture.
Think about trying to get past an HTTP proxy running squid on OpenBSD 
on Sparc in order to attack a web server running Apache on CentOS on x86
in order to attack a DB server running MariaDB...well, you get the idea.)

Third, do you really want your web (et.al.) servers dealing with this?
Aren't they supposed to be providing services, while other systems
block the cruft and deflect attacks?  Yes, you *can* drop all of the
functionality onto one box but that isn't going to scale well and it's
not going to present nearly as hardened a target.  Why not let a much
cheaper box absorb the beating and let your web server provide web services?

Fourth, part of the function of all of these extra servers and services
is to restrict what goes *out* just as much as what comes in.  A backend
DB server might cheerfully be answering thousands of queries per second
as its supposed to, but elsewhere an HTTP proxy farm might be taking note
of an abnormally high amount of outbound traffic headed to a single IP
address and raising an alarm...because that's not right.  Every system
involved should have very strict controls on what traffic it will permit,
where from, where to, what kind, and how much.  (That last one is a
tough sell to folks who always want faster/more.  Faster/more is fine.
Except when it's faster/more data going someplace it shouldn't.  One
of the serious downsides of very high performance environments is that
they perform very well for bad guys as well as good guys.)


I think of this as applying the Software Tools (Kernighan and Plauger)
approach to service design.   A server, like a software tool, should do
one thing and do it well.  Breaking up the tasks of router, firewall,
proxy, web server, database server, etc. and spreading them across
multiple systems (in depth or in parallel or both) not only allows one
to tune each to its task, but it allows very strict control over what
passes over the network connections between them.  Compare/contrast:
how hard would it be to perform data exfiltration against (a) a system
with an onboard firewall running the LAMP stack or (b) a network with
firewall, HTTP proxy, HTTP server, firewall, SQL proxy, DB server?

Yes, it's more complicated and yes, it's more costly.  But consider the
cost of breaches: estimates of the cost of the Target breach are in the
$150M to $200M range.  Estimates of the cost of the Anthem breach are
already at the $100M mark and we're just getting started: I expect that
number to get a lot bigger.  Wouldn't it have been worth dropping $500K
to set up something like this, given how much harder it makes things
for attackers?  (Not that it HAS to cost $500K.  I've set up instances
like this for a lot less.)

And this is not really all that difficult for anyone who has baseline
competency in the core Unix/Linux toolset.  Frankly, the technical issues
are the easy part: that hard part is convincing people to do it, e.g.:

Customer: We'd like to be more secure.
Me: I understand.  Here are preliminary recommendation for changes.
Customer: Changes?  Oh no, we don't want to change anything.  We just
	want to be more secure.
Me: Your underlying architecture is insecure by design.  It cannot be
	patched, modified, band-aided or anything else.  It must be changed.
Customer: We think you should be able to secure it as-is.
Me: (reaches for scotch)
Customer: How about just putting it all the cloud?
Me: (makes it a double)

---rsk



More information about the Novalug mailing list