Discussion:
Domain name based multihome routing?
Dale Mahalko
2018-06-26 05:08:01 UTC
Permalink
(Hello, I am new to the list. And this may possibly be my only post here..)

I am looking for a way on Linux to do domain name based multihome routing.

Essentially every time a domain name lookup request occurs:

* Rather than immediately returning the results to the requesting program,
instead Named/BIND should pause the process and send the results out to a
secondary program.

* The secondary program looks up the domain in a database, which also
includes the multihome destination for each domain. If a match is found, a
route is created to that multihome destination. Aliased acceleration
domains such as Akamai will be matched using the primary domain name.

* Control is now returned to Named/BIND which returns the results as usual
to the original requester. When the secondary program uses the numeric
address(es) returned by Named/BIND, it is routed according to the multhome
destination list.

,

Is there any way to do this with Named/BIND the way it is currently
programmed, or would it be necessary to hack the source to insert this
redirection step?

The specific reason why I need this is that I am one of the many thousands
of rural people in the United States who are stuck on a horribly slow DSL
Internet connection, with a maximum speed of 1.5 megabit down, 0.25 megabit
up, and no way to upgrade. The one redeeming quality of it, is that the
monthly bandwidth is essentially uncapped.

I am looking into buying a second, expensive cellular data plan which
allows 4G speeds of up to about 15 megabit, but which has a monthly data
cap of about 25 gigabytes.

I want to conserve the limited high-speed cellular bandwidth as much as
possible, and put all the downloads that I don't care about on the slow DSL.

* I want to put all the huge background bandwidth eating maintenance
downloads such as Microsoft Windows updates, Microsoft Store updates,
Microsoft P2P updates, Steam game downloads and updates, Adobe updates,
iTunes updates, iPhone iOS and App updates, and so forth on the slow DSL.

* I want to put all the other things that are important to me like
multiplayer gaming UDP streams, remote desktop / SSH, video streaming, and
general web browsing on the cellular modem.

,

Due to there being thousands and thousands of cloud servers, plus bandwidth
optimization services, it is virtually impossible for me to know in advance
and manually/statically route all possible servers that Microsoft, Steam,
Adobe, Apple or any other cloud hosted and Akamai/AWS accelerated business
may use.

In most cases it is not possible to know what newly created cloud servers
these companies will use until the moment they actually request a domain
lookup for that new server within their parent domain.

Hence the multihome routing for these domains must be done dynamically on
the fly, as they are being requested from the name lookup service, but
before the lookup results are returned to the originating program
requesting the lookup.


Dale Mahalko, Gilman, WI, USA
Mark Andrews
2018-06-26 07:23:49 UTC
Permalink
Why send it to a secondary program? Just have named look the name up
in the database directly and then use a route socket to inject the
route. Named already uses a route socket to track interfaces coming
and going.

Note: CDN’s use the same machine for multiple names so you may not always
get the result you are after.

Mark
Post by Dale Mahalko
(Hello, I am new to the list. And this may possibly be my only post here..)
I am looking for a way on Linux to do domain name based multihome routing.
* Rather than immediately returning the results to the requesting program, instead Named/BIND should pause the process and send the results out to a secondary program.
* The secondary program looks up the domain in a database, which also includes the multihome destination for each domain. If a match is found, a route is created to that multihome destination. Aliased acceleration domains such as Akamai will be matched using the primary domain name.
* Control is now returned to Named/BIND which returns the results as usual to the original requester. When the secondary program uses the numeric address(es) returned by Named/BIND, it is routed according to the multhome destination list.
,
Is there any way to do this with Named/BIND the way it is currently programmed, or would it be necessary to hack the source to insert this redirection step?
The specific reason why I need this is that I am one of the many thousands of rural people in the United States who are stuck on a horribly slow DSL Internet connection, with a maximum speed of 1.5 megabit down, 0.25 megabit up, and no way to upgrade. The one redeeming quality of it, is that the monthly bandwidth is essentially uncapped.
I am looking into buying a second, expensive cellular data plan which allows 4G speeds of up to about 15 megabit, but which has a monthly data cap of about 25 gigabytes.
I want to conserve the limited high-speed cellular bandwidth as much as possible, and put all the downloads that I don't care about on the slow DSL.
* I want to put all the huge background bandwidth eating maintenance downloads such as Microsoft Windows updates, Microsoft Store updates, Microsoft P2P updates, Steam game downloads and updates, Adobe updates, iTunes updates, iPhone iOS and App updates, and so forth on the slow DSL.
* I want to put all the other things that are important to me like multiplayer gaming UDP streams, remote desktop / SSH, video streaming, and general web browsing on the cellular modem.
,
Due to there being thousands and thousands of cloud servers, plus bandwidth optimization services, it is virtually impossible for me to know in advance and manually/statically route all possible servers that Microsoft, Steam, Adobe, Apple or any other cloud hosted and Akamai/AWS accelerated business may use.
In most cases it is not possible to know what newly created cloud servers these companies will use until the moment they actually request a domain lookup for that new server within their parent domain.
Hence the multihome routing for these domains must be done dynamically on the fly, as they are being requested from the name lookup service, but before the lookup results are returned to the originating program requesting the lookup.
Dale Mahalko, Gilman, WI, USA
_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list
bind-users mailing list
https://lists.isc.org/mailman/listinfo/bind-users
--
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742 INTERNET: ***@isc.org

_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
bind-***@lists.isc.org
https:
Dale Mahalko
2018-06-26 10:11:31 UTC
Permalink
I should also mention that I am not a formally trained programmer. I am
mostly an end-user looking for a readymade solution that doesn't require
understanding the source and recompiling it.

I can dabble, but I do not know all the intricacies of C/C++ to implement
with any level of stability or quality, of what I am talking about here.
Post by Mark Andrews
Why send it to a secondary program? Just have named look the name up
in the database directly and then use a route socket to inject the
route. Named already uses a route socket to track interfaces coming
and going.
Note: CDN’s use the same machine for multiple names so you may not always
get the result you are after.
Mark
Post by Dale Mahalko
(Hello, I am new to the list. And this may possibly be my only post
here..)
Post by Dale Mahalko
I am looking for a way on Linux to do domain name based multihome
routing.
Post by Dale Mahalko
* Rather than immediately returning the results to the requesting
program, instead Named/BIND should pause the process and send the results
out to a secondary program.
Post by Dale Mahalko
* The secondary program looks up the domain in a database, which also
includes the multihome destination for each domain. If a match is found, a
route is created to that multihome destination. Aliased acceleration
domains such as Akamai will be matched using the primary domain name.
Post by Dale Mahalko
* Control is now returned to Named/BIND which returns the results as
usual to the original requester. When the secondary program uses the
numeric address(es) returned by Named/BIND, it is routed according to the
multhome destination list.
Post by Dale Mahalko
,
Is there any way to do this with Named/BIND the way it is currently
programmed, or would it be necessary to hack the source to insert this
redirection step?
Post by Dale Mahalko
The specific reason why I need this is that I am one of the many
thousands of rural people in the United States who are stuck on a horribly
slow DSL Internet connection, with a maximum speed of 1.5 megabit down,
0.25 megabit up, and no way to upgrade. The one redeeming quality of it, is
that the monthly bandwidth is essentially uncapped.
Post by Dale Mahalko
I am looking into buying a second, expensive cellular data plan which
allows 4G speeds of up to about 15 megabit, but which has a monthly data
cap of about 25 gigabytes.
Post by Dale Mahalko
I want to conserve the limited high-speed cellular bandwidth as much as
possible, and put all the downloads that I don't care about on the slow DSL.
Post by Dale Mahalko
* I want to put all the huge background bandwidth eating maintenance
downloads such as Microsoft Windows updates, Microsoft Store updates,
Microsoft P2P updates, Steam game downloads and updates, Adobe updates,
iTunes updates, iPhone iOS and App updates, and so forth on the slow DSL.
Post by Dale Mahalko
* I want to put all the other things that are important to me like
multiplayer gaming UDP streams, remote desktop / SSH, video streaming, and
general web browsing on the cellular modem.
Post by Dale Mahalko
,
Due to there being thousands and thousands of cloud servers, plus
bandwidth optimization services, it is virtually impossible for me to know
in advance and manually/statically route all possible servers that
Microsoft, Steam, Adobe, Apple or any other cloud hosted and Akamai/AWS
accelerated business may use.
Post by Dale Mahalko
In most cases it is not possible to know what newly created cloud
servers these companies will use until the moment they actually request a
domain lookup for that new server within their parent domain.
Post by Dale Mahalko
Hence the multihome routing for these domains must be done dynamically
on the fly, as they are being requested from the name lookup service, but
before the lookup results are returned to the originating program
requesting the lookup.
Post by Dale Mahalko
Dale Mahalko, Gilman, WI, USA
_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to
unsubscribe from this list
Post by Dale Mahalko
bind-users mailing list
https://lists.isc.org/mailman/listinfo/bind-users
--
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
Grant Taylor via bind-users
2018-06-26 17:45:22 UTC
Permalink
Post by Dale Mahalko
* The secondary program looks up the domain in a database, which also
includes the multihome destination for each domain. If a match is found,
a route is created to that multihome destination. Aliased acceleration
domains such as Akamai will be matched using the primary domain name.
Are you saying that you want to dynamically update routes to IPs
resolved in real time to specific host / domain names? Such that
traffic to specific hosts / domain names is routed over DSL? With
things that don't match conditions routed over cell?
Post by Dale Mahalko
* I want to put all the huge background bandwidth eating maintenance
downloads such as Microsoft Windows updates, Microsoft Store updates,
Microsoft P2P updates, Steam game downloads and updates, Adobe updates,
iTunes updates, iPhone iOS and App updates, and so forth on the slow DSL.
* I want to put all the other things that are important to me like
multiplayer gaming UDP streams, remote desktop / SSH, video streaming,
and general web browsing on the cellular modem.
I think I understand what you want to do and why you want to do it.

It seems like you're using named as the source of information to feed
into the process that dynamically updates routing.

I find the pausing of named to be questionable. But I understand that
you want to make sure that no connections are started until after the
(re)routing has been done.

I feel like most of this is outside of named's scope and that it would
run as a different user.

I would suggest exploring BIND's new Response Policy Service. I think
it provides a way for BIND to send information to a side program for
various ""filtering actions. IMHO there's no reason that such a side
program has to actually filter requests / responses. Instead, you could
use that as an information feed to do what you're wanting to do with IPs
and routes. I just don't know about the ability to pause the response.
Unless it's possible to do the route modification before returning the
reply to BIND.
--
Grant. . . .
unix || die
Warren Kumari
2018-06-26 18:26:53 UTC
Permalink
On Tue, Jun 26, 2018 at 12:45 PM Grant Taylor via bind-users <
Post by Grant Taylor via bind-users
Post by Dale Mahalko
* The secondary program looks up the domain in a database, which also
includes the multihome destination for each domain. If a match is found,
a route is created to that multihome destination. Aliased acceleration
domains such as Akamai will be matched using the primary domain name.
Are you saying that you want to dynamically update routes to IPs
resolved in real time to specific host / domain names? Such that
traffic to specific hosts / domain names is routed over DSL? With
things that don't match conditions routed over cell?
It feels like one should be able to cobble together something hilarious
using:
A: RPZ to return a AAAA only answer,
B: NAT64

Have RPZ suppress the A record, and return a synthesized AAAA with the
NAT64 prefix tacked on the front. This will route it to a NAT64 box which
converts it to a v4 address, and Bobs yer uncle.

This seems like it would work, but be fragile and annoying.

W
Post by Grant Taylor via bind-users
Post by Dale Mahalko
* I want to put all the huge background bandwidth eating maintenance
downloads such as Microsoft Windows updates, Microsoft Store updates,
Microsoft P2P updates, Steam game downloads and updates, Adobe updates,
iTunes updates, iPhone iOS and App updates, and so forth on the slow DSL.
* I want to put all the other things that are important to me like
multiplayer gaming UDP streams, remote desktop / SSH, video streaming,
and general web browsing on the cellular modem.
I think I understand what you want to do and why you want to do it.
It seems like you're using named as the source of information to feed
into the process that dynamically updates routing.
I find the pausing of named to be questionable. But I understand that
you want to make sure that no connections are started until after the
(re)routing has been done.
I feel like most of this is outside of named's scope and that it would
run as a different user.
I would suggest exploring BIND's new Response Policy Service. I think
it provides a way for BIND to send information to a side program for
various ""filtering actions. IMHO there's no reason that such a side
program has to actually filter requests / responses. Instead, you could
use that as an information feed to do what you're wanting to do with IPs
and routes. I just don't know about the ability to pause the response.
Unless it's possible to do the route modification before returning the
reply to BIND.
--
Grant. . . .
unix || die
_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to
unsubscribe from this list
bind-users mailing list
https://lists.isc.org/mailman/listinfo/bind-users
--
I don't think the execution is relevant when it was obviously a bad idea in
the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair of
pants.
---maf
Dale Mahalko
2018-06-26 19:07:02 UTC
Permalink
On Tue, Jun 26, 2018 at 12:45 PM, Grant Taylor via bind-users <
Are you saying that you want to dynamically update routes to IPs resolved
in real time to specific host / domain names? Such that traffic to
specific hosts / domain names is routed over DSL? With things that don't
match conditions routed over cell?
Yes.


I think I understand what you want to do and why you want to do it.
It seems like you're using named as the source of information to feed into
the process that dynamically updates routing.
I find the pausing of named to be questionable. But I understand that you
want to make sure that no connections are started until after the
(re)routing has been done.
(I am no programming expert as mentioned, but I do IT stuff for a living,
so..)

The pause would only be long enough to look for a regex domain pattern to
be routed to the DSL, and then creating the route. This pause can likely be
measured in nanoseconds.

This would likely be a multithreaded asynchronous mechanism so that BIND
does each of its lookups as usual, and then forks a followup thread after
it completes its normal lookup process, to do the pattern match and route
creation, followed by the delayed response released when the
pattern-match/route-creation thread terminates.

So in general using multithreading, there would be no real impact to
programs requesting the lookups, other than a delay per lookup that is so
small it would not be noticeable to an end-user human.
Matus UHLAR - fantomas
2018-06-27 14:16:16 UTC
Permalink
Post by Dale Mahalko
On Tue, Jun 26, 2018 at 12:45 PM, Grant Taylor via bind-users <
Are you saying that you want to dynamically update routes to IPs resolved
in real time to specific host / domain names? Such that traffic to
specific hosts / domain names is routed over DSL? With things that don't
match conditions routed over cell?
I think I understand what you want to do and why you want to do it.
It seems like you're using named as the source of information to feed into
the process that dynamically updates routing.
I find the pausing of named to be questionable. But I understand that you
want to make sure that no connections are started until after the
(re)routing has been done.
(I am no programming expert as mentioned, but I do IT stuff for a living,
so..)
The pause would only be long enough to look for a regex domain pattern to
be routed to the DSL, and then creating the route. This pause can likely be
measured in nanoseconds.
I don't think this could be done in nanoseconds. Maybe microseconds, but
more probably miliseconds.

Another question would be, how fast your router can be with potentially
thousands of routes (I know, many OSes have routing optimised very hardly).
Post by Dale Mahalko
This would likely be a multithreaded asynchronous mechanism so that BIND
does each of its lookups as usual, and then forks a followup thread after
it completes its normal lookup process, to do the pattern match and route
creation, followed by the delayed response released when the
pattern-match/route-creation thread terminates.
So in general using multithreading, there would be no real impact to
programs requesting the lookups, other than a delay per lookup that is so
small it would not be noticeable to an end-user human.
I think that you are trying wrong approach, using wrong tools.
Guessing the potential usage from DNS is not a goog idea.

On your router, configure firewall to route selected protocols (gaming, ssh,
RDP, dns) and maybe later some sites to paid cellular and router everything
other to DSL.

Note that at my home, most of data is spend by my children watching youtube
videos - I don't think that routing general web and streaming services to
cell connection would help you with anything.
--
Matus UHLAR - fantomas, ***@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
M$ Win's are shit, do not use it !
_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
bind-***@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users
Dale Mahalko
2018-06-27 16:59:47 UTC
Permalink
There is no way to know if this is the "right" or "wrong" approach without
actually trying it and see what happens.

Guessing the potential background domains used by Microsoft / Steam, etc
and monitoring bandwidth used by those domains is unfortunately the only
option available. It's not like any of these companies are willing to
outright divulge anything about these background details to anyone outside
their business.

As far as load on the router goes for keeping track of possibly tens of
thousands of custom routes, I am fine with dedicating a fast Intel i5 or i7
and a couple gigabytes of memory to the job. Most routers are tiny little
things with very little CPU needed for normal routing, with the heavy
lifting only happening if encryption is needed for a bunch of VPN
connections.
Post by Dale Mahalko
On Tue, Jun 26, 2018 at 12:45 PM, Grant Taylor via bind-users <
Post by Dale Mahalko
Are you saying that you want to dynamically update routes to IPs resolved
in real time to specific host / domain names? Such that traffic to
specific hosts / domain names is routed over DSL? With things that don't
match conditions routed over cell?
I think I understand what you want to do and why you want to do it.
It seems like you're using named as the source of information to feed into
Post by Dale Mahalko
the process that dynamically updates routing.
I find the pausing of named to be questionable. But I understand that you
want to make sure that no connections are started until after the
(re)routing has been done.
(I am no programming expert as mentioned, but I do IT stuff for a living,
so..)
The pause would only be long enough to look for a regex domain pattern to
be routed to the DSL, and then creating the route. This pause can likely be
measured in nanoseconds.
I don't think this could be done in nanoseconds. Maybe microseconds, but
more probably miliseconds.
Another question would be, how fast your router can be with potentially
thousands of routes (I know, many OSes have routing optimised very hardly).
This would likely be a multithreaded asynchronous mechanism so that BIND
Post by Dale Mahalko
does each of its lookups as usual, and then forks a followup thread after
it completes its normal lookup process, to do the pattern match and route
creation, followed by the delayed response released when the
pattern-match/route-creation thread terminates.
So in general using multithreading, there would be no real impact to
programs requesting the lookups, other than a delay per lookup that is so
small it would not be noticeable to an end-user human.
I think that you are trying wrong approach, using wrong tools.
Guessing the potential usage from DNS is not a goog idea.
On your router, configure firewall to route selected protocols (gaming, ssh,
RDP, dns) and maybe later some sites to paid cellular and router everything
other to DSL.
Note that at my home, most of data is spend by my children watching youtube
videos - I don't think that routing general web and streaming services to
cell connection would help you with anything.
--
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
M$ Win's are shit, do not use it !
_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to
unsubscribe from this list
bind-users mailing list
https://lists.isc.org/mailman/listinfo/bind-users
Darcy Kevin (FCA)
2018-06-27 17:27:52 UTC
Permalink
I’m not convinced DNS has any valuable role to play here. Seems like this is a traffic-shaping challenge; maybe one of the open source traffic shaping tools would fit the bill.

- Kevin


From: bind-users <bind-users-***@lists.isc.org> On Behalf Of Dale Mahalko
Sent: Wednesday, June 27, 2018 1:00 PM
To: bind-***@lists.isc.org
Subject: Re: Domain name based multihome routing?

There is no way to know if this is the "right" or "wrong" approach without actually trying it and see what happens.

Guessing the potential background domains used by Microsoft / Steam, etc and monitoring bandwidth used by those domains is unfortunately the only option available. It's not like any of these companies are willing to outright divulge anything about these background details to anyone outside their business.

As far as load on the router goes for keeping track of possibly tens of thousands of custom routes, I am fine with dedicating a fast Intel i5 or i7 and a couple gigabytes of memory to the job. Most routers are tiny little things with very little CPU needed for normal routing, with the heavy lifting only happening if encryption is needed for a bunch of VPN connections.

On Wed, Jun 27, 2018 at 9:16 AM, Matus UHLAR - fantomas <***@fantomas.sk<mailto:***@fantomas.sk>> wrote:
On Tue, Jun 26, 2018 at 12:45 PM, Grant Taylor via bind-users <
bind-***@lists.isc.org<mailto:bind-***@lists.isc.org>> wrote:
Are you saying that you want to dynamically update routes to IPs resolved
in real time to specific host / domain names? Such that traffic to
specific hosts / domain names is routed over DSL? With things that don't
match conditions routed over cell?

I think I understand what you want to do and why you want to do it.

It seems like you're using named as the source of information to feed into
the process that dynamically updates routing.

I find the pausing of named to be questionable. But I understand that you
want to make sure that no connections are started until after the
(re)routing has been done.

On 26.06.18 14:07, Dale Mahalko wrote:
(I am no programming expert as mentioned, but I do IT stuff for a living,
so..)

The pause would only be long enough to look for a regex domain pattern to
be routed to the DSL, and then creating the route. This pause can likely be
measured in nanoseconds.

I don't think this could be done in nanoseconds. Maybe microseconds, but
more probably miliseconds.

Another question would be, how fast your router can be with potentially
thousands of routes (I know, many OSes have routing optimised very hardly).
This would likely be a multithreaded asynchronous mechanism so that BIND
does each of its lookups as usual, and then forks a followup thread after
it completes its normal lookup process, to do the pattern match and route
creation, followed by the delayed response released when the
pattern-match/route-creation thread terminates.

So in general using multithreading, there would be no real impact to
programs requesting the lookups, other than a delay per lookup that is so
small it would not be noticeable to an end-user human.

I think that you are trying wrong approach, using wrong tools.
Guessing the potential usage from DNS is not a goog idea.

On your router, configure firewall to route selected protocols (gaming, ssh,
RDP, dns) and maybe later some sites to paid cellular and router everything
other to DSL.

Note that at my home, most of data is spend by my children watching youtube
videos - I don't think that routing general web and streaming services to
cell connection would help you with anything.
--
Matus UHLAR - fantomas, ***@fantomas.sk<mailto:***@fantomas.sk> ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
M$ Win's are shit, do not use it !

_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
bind-***@lists.isc.org<mailto:bind-***@lists.isc.org>
https://lists.isc.org/mailman/listinfo/bind-users
Dale Mahalko
2018-06-27 18:17:41 UTC
Permalink
On Wed, Jun 27, 2018 at 12:27 PM, Darcy Kevin (FCA) <
Post by Darcy Kevin (FCA)
I’m not convinced DNS has any valuable role to play here. Seems like this
is a traffic-shaping challenge; maybe one of the open source traffic
shaping tools would fit the bill.
A Google search for multihome traffic shaping yields nothing obvious.

Do you have specific details you can share about exactly how that would be
done?

Also how is traffic shaping going to tell the difference between a
background Apple iOS update or Windows update that need to use the DSL, and
the high priority data streams that are more important to me, that need to
use the cellular modem?


Shaping is not routing, it just prioritizes some data streams over others.
I don't see how shaping is going to know whether to use the DSL or the
Cellular ... without inspecting the domain name before a connection is
established.... which is what I'm already discussing here...
Darcy Kevin (FCA)
2018-06-27 18:45:42 UTC
Permalink
Traffic shaping is not my area of expertise, but from what I understand, at a minimum it can classify different kinds of traffic, based on more reliable metrics than DNS name. I was assuming (perhaps incorrectly), that its output (QoS markings or CoS or whatever) could then be used in a degenerate mode to force certain types of traffic over particular WAN connections, by manipulating costs, thresholds, etc.

In a quick scan, I found this article https://turbofuture.com/computers/How-to-Configure-Deep-Packet-Inspection-Using-pfSense (URL is misleading; the vast majority of the article isn’t about DPI at all). This shows a pfSense “wizard” that generates different profiles depending on your particular combination of single/multiple WANs and/or LANs. What I take from the guide is that the traffic shaping can know about your WAN setup and can be tweaked to push the traffic the way you want it to, over different WAN links.

I might be completely off-base on this, but it seems like a more fruitful line of research/inquiry than determining traffic profiles based on DNS names, and then hacking BIND to manipulate your routing table on-the-fly. That seems to me fraught with challenges, risks and limitations.

- Kevin


From: Dale Mahalko <***@gmail.com>
Sent: Wednesday, June 27, 2018 2:18 PM
To: Darcy Kevin (FCA) <***@fcagroup.com>
Cc: bind-***@lists.isc.org
Subject: Re: Domain name based multihome routing?

On Wed, Jun 27, 2018 at 12:27 PM, Darcy Kevin (FCA) <***@fcagroup.com<mailto:***@fcagroup.com>> wrote:
I’m not convinced DNS has any valuable role to play here. Seems like this is a traffic-shaping challenge; maybe one of the open source traffic shaping tools would fit the bill.

A Google search for multihome traffic shaping yields nothing obvious.

Do you have specific details you can share about exactly how that would be done?

Also how is traffic shaping going to tell the difference between a background Apple iOS update or Windows update that need to use the DSL, and the high priority data streams that are more important to me, that need to use the cellular modem?


Shaping is not routing, it just prioritizes some data streams over others. I don't see how shaping is going to know whether to use the DSL or the Cellular ... without inspecting the domain name before a connection is established.... which is what I'm already discussing here...
Paul Kosinski
2018-06-27 19:41:09 UTC
Permalink
We do something somewhat similar with our LAN. We have a new cable
connection and an old DSL connection. The cable is 60x faster, but has
a dynamic IP and blocks various ports (esp. 25), so we keep the DSL so
we can send email directly etc.

Obviously, we don't want to stream video or even do much Web browsing
over the DSL. So we have set up a Linux computer to serve as a gateway
and firewall: it runs IPtables, Privoxy, HAVP (virus filter for HTTP),
ClamAV and even Bind (a 3rd DNS server for our small domains).

This works fairly straightforwardly because decision as to whether to
use cable or DSL is made according to the *source* IP address, rather
than the destination IP address (or domain name, or port). Since
many browsers (we use Firefox) and other Internet software have the
ability to specify a proxy for Internet access, we usually connect them
to a proxy server on the gateway which in turn binds to an alias IP on
either the NIC connected to the DSL modem or the cable modem.

Then we have 2 routing tables, the default one for the (original) DSL
and a second one for cable. Each routing table gas its own default
route, and each is 'via' the corresponding modem. To decide which way
packets go, we make use of a 'rule' table (iproute2) which says which
routing table to use. It has entries generated by iproute2 functions
such as:

/sbin/ip rule add from <alias IP> lookup cable
/sbin/ip rule add to <cable modem control IP> lookup cable
/sbin/ip rule add iif br2 lookup cable

This last rule says the *everything* from (sub) LAN 2 goes via cable.
This allows whole sets of devices (such as our computer dedicated to TV)
to be connected strictly to cable.

Note that even though you bind to an alias IP on the NIC physically
connected to a specific modem, if that modem isn't the overall default
route, you still need a 'rule' to make the kernel do the right thing.

In summary, this scheme does not give you totally automatic control of
what kind of traffic goes by what physical link, but it does allow
different browser instances on a single computer to use different
physical links via proxying, plus it easily allows different devices on
the LAN to be handled differently (since they each have their own IP
address).

----------------------------------

On Wed, 27 Jun 2018 13:17:41 -0500
Post by Dale Mahalko
On Wed, Jun 27, 2018 at 12:27 PM, Darcy Kevin (FCA) <
I’m not convinced DNS has any valuable role to play here. Seems
like this is a traffic-shaping challenge; maybe one of the open
source traffic shaping tools would fit the bill.
A Google search for multihome traffic shaping yields nothing obvious.
Do you have specific details you can share about exactly how that
would be done?
Also how is traffic shaping going to tell the difference between a
background Apple iOS update or Windows update that need to use the
DSL, and the high priority data streams that are more important to
me, that need to use the cellular modem?
Shaping is not routing, it just prioritizes some data streams over
others. I don't see how shaping is going to know whether to use the
DSL or the Cellular ... without inspecting the domain name before a
connection is established.... which is what I'm already discussing
here...
_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
bind-***@lists.isc.
Grant Taylor via bind-users
2018-06-28 03:03:45 UTC
Permalink
Post by Darcy Kevin (FCA)
I’m not convinced DNS has any valuable role to play here.
I can see the value for services that have FQDNs that resolve to IP addresses outside of their ASN(s) like Google / YouTube.
--
Grant. . . .
unix || die
Darcy Kevin (FCA)
2018-06-28 22:04:36 UTC
Permalink
Yeah, but it's not an exact science, any way you slice it.

I just did a quick crunch of yesterday's data from our web proxy logs, and accesses of URIs based on the FQDN "b.scorecardresearch.com" (a banner ad site, I believe) had over 570 different combinations of website content categories, depending on URI. One FQDN, 570 different possible ways one might want to direct the traffic. DNS-based approaches simply may not have the granularity necessary to get the job done.

Speaking of web proxies, that should probably be the *first* thing that gets put into place, if the goal is minimize "disfavored" web traffic from traversing expensive WAN connections.

- Kevin


-----Original Message-----
From: bind-users <bind-users-***@lists.isc.org> On Behalf Of Grant Taylor via bind-users
Sent: Wednesday, June 27, 2018 11:04 PM
Cc: bind-***@lists.isc.org
Subject: Re: Domain name based multihome routing?
I’m not convinced DNS has any valuable role to play here.
I can see the value for services that have FQDNs that resolve to IP addresses outside of their ASN(s) like Google / YouTube.



--
Grant. . . .
unix || die
_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
bind-***@lists.isc.org
https://lists.isc.org/mailman/listi
Dale Mahalko
2018-06-29 00:50:27 UTC
Permalink
Eh, I gave up on web proxies a couple years back where I work. It is mostly
pointless in the age of "SSL for everything" after Snowden spilled the
beans on US gov spying of all open traffic. I am not interested in the
complexities of MITM certificates that web browsers are going to constantly
scream about.

Also it is highly unclear to me if the more recent P2P update mechanisms
from Microsoft or Steam use HTTP at all anymore, so proxying may be mostly
useless for those largest of bandwidth consumers.

How is proxying somehow easier than just checking a domain to see if it
needs special routing with each DNS request, routing the resulting numeric
address(es) if there is a match, and then not needing to hack my system
security or needing to be intimately knowledgeable of whatever
communications protocols are used beyond that point over the specially
assigned route?

Grant Taylor via bind-users
2018-06-28 03:02:03 UTC
Permalink
Post by Darcy Kevin (FCA)
Guessing the potential background domains used by Microsoft / Steam, etc and monitoring bandwidth used by those domains is unfortunately the only option available.
If you can get information on the IP addresses associated with their ASN(s) you could route them out the DSL connection.

This might not work well for Google / YouTube or any other service that uses IPs outside of their ASNs.
--
Grant. . . .
unix || die
Dale Mahalko
2018-06-27 18:00:24 UTC
Permalink
Due to the fact that I don't have the ability to program this experiment
myself without spending a couple more years to improve my coding skills,
could I interest anyone else here to do the programming work?

I would prefer someone who is associated with ISC who sounds like they
already know the code, like Mark Andrews.

I would pay for your time on this, and the results would be free open
source for anyone else to use.. and could be included as an extension of
the standard code if the maintainers would allow it.

Though if you want more than about US$500 for your efforts, then I will
probably have to try to get others involved on a crowdfunding website to
cover the costs.

Dale Mahalko, Gilman, WI



Living on a rural 35-cow organic dairy farm, ten miles from the nearest
town, on a slow CenturyLink 1.5 meg DSL and no way to upgrade.

The CenturyLink remote terminal near us has been "in exhaust" for the last
15 years, and they are unwilling to install the necessary 10 mile / 16 km
fiber backhaul to their DSLAM cabinet, even though we are in an area that
qualifies for Connect America Fund - Phase II (CAF-II) funding assistance
from the federal government to get the fiber installed.

CenturyLink has discretion to "divert" the CAF-II funds to other things if
they want and it appears that has happened, so we will remain trapped with
this poor level of landline service unless I go to extremes to try to find
something better.

I get about 2-3 bars on the iPhone, so I am preparing to spend about $600
on a MOFI 4500 cell modem and some huge outdoor dual-MIMO yagi WirEng
cellular modem antennas to go on the roof of the house to boost the signal.

(Satellite is unacceptable. I require low latency for remote desktop, work
from home, gaming, etc.)
Loading...