dig with and without +norec

[Ladislav Vobr]

I am somehow resending my earlier post. Can somebody give a hint why
recursive caching server behaves like this.

I have issued the dig directly from the server.

[dxbins1:/]#dig ns af.mil @192.168.8.91

; <<>> DiG 9.2.3 <<>> ns af.mil @192.168.8.91
;; global options: printcmd
;; connection timed out; no servers could be reached


[dxbins1:/]#dig ns af.mil @192.168.8.91 +norec

; <<>> DiG 9.2.3 <<>> ns af.mil @192.168.8.91 +norec
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 60916
;; flags: qr ra; QUERY: 1, ANSWER: 0, AUTHORITY: 6, ADDITIONAL: 0

;; QUESTION SECTION:
;af.mil. IN NS

;; AUTHORITY SECTION:
af.mil. 58999 IN NS ARTEMIS.AFNOC.af.mil.
af.mil. 58999 IN NS NS.USAFE.af.mil.
af.mil. 58999 IN NS NS.MAXWELL.af.mil.
af.mil. 58999 IN NS MARS.AFNOC.af.mil.
af.mil. 58999 IN NS PAPA1.BARKSDALE.af.mil.
af.mil. 58999 IN NS DELTA1.BARKSDALE.af.mil.

;; Query time: 19 msec
;; SERVER: 192.168.8.91#53(192.168.8.91)
;; WHEN: Sat Feb 28 08:12:24 2004
;; MSG SIZE rcvd: 170

[Barry Margolin]

In article <c1p590$v89$1 at sf1.isc.org>,

Post by Ladislav Vobr
I am somehow resending my earlier post. Can somebody give a hint why
recursive caching server behaves like this.

When you specify +norec, it answers with whatever it has in its cache,
which in this case is the delegation records that came from the parent
domain's servers.

When you request recursion, it checks whether the cached information
came from an authoritative server. If not, it tries to ask the
authoritative servers.

I'm not sure why the recursive query is failing for you, though. I have
no problem querying any of the authoritative nameservers. However, it's
notable that your second dig returned the NS records in the Authority
Section rather than the Answer Section, and didn't include the glue A
records in the Additional Section. What version of BIND are you running
on the server?

Post by Ladislav Vobr
I have issued the dig directly from the server.
;; global options: printcmd
;; connection timed out; no servers could be reached
;; global options: printcmd
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 60916
;; flags: qr ra; QUERY: 1, ANSWER: 0, AUTHORITY: 6, ADDITIONAL: 0
;af.mil. IN NS
af.mil. 58999 IN NS ARTEMIS.AFNOC.af.mil.
af.mil. 58999 IN NS NS.USAFE.af.mil.
af.mil. 58999 IN NS NS.MAXWELL.af.mil.
af.mil. 58999 IN NS MARS.AFNOC.af.mil.
af.mil. 58999 IN NS PAPA1.BARKSDALE.af.mil.
af.mil. 58999 IN NS DELTA1.BARKSDALE.af.mil.
;; Query time: 19 msec
;; SERVER: 192.168.8.91#53(192.168.8.91)
;; WHEN: Sat Feb 28 08:12:24 2004
;; MSG SIZE rcvd: 170

[Ladislav Vobr]

Barry,

Thanks for a hint, I have discovered I am being blocked by them
probably, I don't know why, but it is us air-force, perhaps middle east
should not see them :-)... I am trying to contact them to investigate.

I have 9.2.3, I though this is happening and bind although having it in
the cache is trying to verify with the authoritative. It looks like I
have even performance problem because of this, since the same servers
are hosting lot of zones especially reverse ones, and retrying to them
takes some resources out of my bind.

Will it answer from cache if I bogus all af.mil ns?

In bind9 I don't see anymore from dump_db what is the ip address of the
server I got this cached data from, or what is the level of the trust of
the cached data. It is hard to troubleshoot, when you don't know who
sent you this cache and how much bind9 trust it.

Does it mean that if I have an A record in the cache, which I got from
the parent as a glue and the authoritative server is unreachable, bind
will not answer any recursive request about this record?


Ladislav

Post by Barry Margolin
In article <c1p590$v89$1 at sf1.isc.org>,

Post by Ladislav Vobr
I am somehow resending my earlier post. Can somebody give a hint why
recursive caching server behaves like this.

When you specify +norec, it answers with whatever it has in its cache,
which in this case is the delegation records that came from the parent
domain's servers.
When you request recursion, it checks whether the cached information
came from an authoritative server. If not, it tries to ask the
authoritative servers.
I'm not sure why the recursive query is failing for you, though. I have
no problem querying any of the authoritative nameservers. However, it's
notable that your second dig returned the NS records in the Authority
Section rather than the Answer Section, and didn't include the glue A
records in the Additional Section. What version of BIND are you running
on the server?

Post by Ladislav Vobr
I have issued the dig directly from the server.
;; global options: printcmd
;; connection timed out; no servers could be reached
;; global options: printcmd
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 60916
;; flags: qr ra; QUERY: 1, ANSWER: 0, AUTHORITY: 6, ADDITIONAL: 0
;af.mil. IN NS
af.mil. 58999 IN NS ARTEMIS.AFNOC.af.mil.
af.mil. 58999 IN NS NS.USAFE.af.mil.
af.mil. 58999 IN NS NS.MAXWELL.af.mil.
af.mil. 58999 IN NS MARS.AFNOC.af.mil.
af.mil. 58999 IN NS PAPA1.BARKSDALE.af.mil.
af.mil. 58999 IN NS DELTA1.BARKSDALE.af.mil.
;; Query time: 19 msec
;; SERVER: 192.168.8.91#53(192.168.8.91)
;; WHEN: Sat Feb 28 08:12:24 2004
;; MSG SIZE rcvd: 170

[Jim Reid]

Ladislav> I have issued the dig directly from the server.

Ladislav> [dxbins1:/]#dig ns af.mil @192.168.8.91

Ladislav> ;; connection timed out; no servers could be reached

Ladislav> [dxbins1:/]#dig ns af.mil @192.168.8.91 +norec

Ladislav> ... referral answer snipped ....

There will probably be a firewall or router in front of 192.168.8.91
that's blocking recursive DNS queries. This would not be unreasonable
if the administrator of 192.168.8.91 didn't want that server to handle
recursive DNS queries.

[Ladislav Vobr]

Post by Jim Reid
Ladislav> ... referral answer snipped ....
There will probably be a firewall or router in front of 192.168.8.91
that's blocking recursive DNS queries. This would not be unreasonable
if the administrator of 192.168.8.91 didn't want that server to handle
recursive DNS queries.

jim, that administrator is me :-), there is a pix firewal, but I don't
have problem answering other recursive queries, and I don't have local
network problems. We have very redundant network setup here, and BTW
f-root in next room. I have around 1000 requests/sec normal traffic
working fine.

For some reasons all af.mil ns servers are unreachable for me. I guess
we have been blocked somewhere and investigating it...

but what was not clear to me is that my recursive bind 9.2.3 have the
data in the cache as a glue from parent server (.mil), but doesn't
provide them for recursive client, unless authoritative servers are up,
but provides them without this check to the non-recursive one.

This is what I observe here, why is that like this or what use it could
have? - right now I don't know. Perhaps it is good to verify with
authoritative server to prevent dns spoofing, or it some kind of favor
bind will do to provide more accurate information, but in my case I am
quite sure it is not spoofed and instead of favor, I would rather accept
the glue as a reply when having a choice (glue or no reply)but I don't
have any way how to provide the answer to my recursive clients, even
though I have it in the cache.

More I think about it, I understand glue is not an answer, but bind
already trust it itself, when it accepted it from the parent.

I have increased load on this server, I am keeping retrying to these
af.mil servers, and I can not even find out if there are any other ip
addresses being retried like this from the same server although they are
unreachable for perhaps days,weeks. There is no log, which tells you
these servers are unreachable I am waisting 5% of resources on each by
flooding them, because bind doesn't consider them lame and it does not
even log it anywhere or inform about it. I can mark them bogus, but how
I discover them?, just by snoop:-) why bind cannot tell look, this guy
is persistently down I will not waste my time/resources on it I will log
it for you or I will blackhole it temporarily if you configure me to do
so. This problems are not issues when you run 100-200 req/second it will
somehow work, but when you go to thousands/sec you need quite a server
for the current behaviour.


Ladislav

p.s. I apologize for what I said in my previous post, that bind9 doesn't
have Cr tag in the named_dump.db, it is there just on the next line, I
used to do grep with bind8, which doesn't show it anymore. But the
source of the cached information (ip address of the parent) is not
there, and this makes troubleshooting difficult.

[Simon Waters]

Post by Ladislav Vobr

Post by Jim Reid
Ladislav> ... referral answer snipped ....
There will probably be a firewall or router in front of 192.168.8.91
that's blocking recursive DNS queries. This would not be unreasonable
if the administrator of 192.168.8.91 didn't want that server to handle
recursive DNS queries.

jim, that administrator is me :-), there is a pix firewal, but I don't
have problem answering other recursive queries, and I don't have local
network problems.

Jim might be right for the wrong reason.

Post by Ladislav Vobr
From here I get ID mismatch querying any of the af.mil servers, which I

would expect a good firewall to toss out and possibly log, since it
suggests a spoofing attack. In the absence of a good firewall BIND 9
happily worries about them instead.

Broken DNS servers(?) - now whether they are deliberately broken is the
question, but my guess is some misguided load balancing or routing hack
- of course someone could be trying to spoof af.mil but I think that the
least likely explanation, as spoofers would do a better job.

Since I'm using a UK ISP connection - doesn't look like a conspiracy
against the middle east - unless the Whitehouse are very upset with
Claire Short ;)

$ dig @198.220.211.145 af.mil ns
;; reply from unexpected source: 127.0.0.1#53, expected 198.220.211.145#53
;; Warning: ID mismatch: expected ID 35420, got 12807
;; reply from unexpected source: 127.0.0.1#53, expected 198.220.211.145#53
;; Warning: ID mismatch: expected ID 35420, got 12807

Traceroutes don't shed any light on the routing, but in good traditional
Internet style my queries to EUR1.NIPR.MIL route via New York, maybe I'm
reading too much into the "EUR".


-- Attached file included as plaintext by Ecartis --
-- File: signature.asc
-- Desc: OpenPGP digital signature

[lauren]

I am assuming you work for an ISP... I would bet that one
of your customers was attempting to do something nasty on
a US Air Force system and tripped one or more of the IDSs.
Therefore your entire block of IPs probably got blocked
at the AF perimeter.

Post by Ladislav Vobr

Post by Ladislav Vobr

Post by Jim Reid
Ladislav> ... referral answer snipped ....
There will probably be a firewall or router in front

of

Post by Ladislav Vobr
192.168.8.91

Post by Jim Reid
that's blocking recursive DNS queries. This would not

be unreasonable

Post by Jim Reid
if the administrator of 192.168.8.91 didn't want that

server to handle

Post by Jim Reid
recursive DNS queries.

jim, that administrator is me :-), there is a pix
firewal, but I don't
have problem answering other recursive queries, and I
don't have local
network problems. We have very redundant network setup
here, and BTW
f-root in next room. I have around 1000 requests/sec
normal traffic
working fine.
For some reasons all af.mil ns servers are unreachable
for me. I guess
we have been blocked somewhere and investigating it...
but what was not clear to me is that my recursive bind
9.2.3 have the
data in the cache as a glue from parent server (.mil),
but doesn't
provide them for recursive client, unless authoritative
servers are up,
but provides them without this check to the

non-recursive

Post by Ladislav Vobr
one.
This is what I observe here, why is that like this or
what use it could
have? - right now I don't know. Perhaps it is good to
verify with
authoritative server to prevent dns spoofing, or it

some

Post by Ladislav Vobr
kind of favor
bind will do to provide more accurate information, but

in

Post by Ladislav Vobr
my case I am
quite sure it is not spoofed and instead of favor, I
would rather accept
the glue as a reply when having a choice (glue or no
reply)but I don't
have any way how to provide the answer to my recursive
clients, even
though I have it in the cache.
More I think about it, I understand glue is not an
answer, but bind
already trust it itself, when it accepted it from the
parent.
I have increased load on this server, I am keeping
retrying to these
af.mil servers, and I can not even find out if there

are

Post by Ladislav Vobr
any other ip
addresses being retried like this from the same server
although they are
unreachable for perhaps days,weeks. There is no log,
which tells you
these servers are unreachable I am waisting 5% of
resources on each by
flooding them, because bind doesn't consider them lame
and it does not
even log it anywhere or inform about it. I can mark

them

Post by Ladislav Vobr
bogus, but how
I discover them?, just by snoop:-) why bind cannot tell
look, this guy
is persistently down I will not waste my time/resources
on it I will log
it for you or I will blackhole it temporarily if you
configure me to do
so. This problems are not issues when you run 100-200
req/second it will
somehow work, but when you go to thousands/sec you need
quite a server
for the current behaviour.
Ladislav
p.s. I apologize for what I said in my previous post,
that bind9 doesn't
have Cr tag in the named_dump.db, it is there just on

the

Post by Ladislav Vobr
next line, I
used to do grep with bind8, which doesn't show it
anymore. But the
source of the cached information (ip address of the
parent) is not
there, and this makes troubleshooting difficult.

__________________________________
Do you Yahoo!?
Get better spam protection with Yahoo! Mail.
http://antispam.yahoo.com/tools

__________________________________
Do you Yahoo!?
Get better spam protection with Yahoo! Mail.
http://antispam.yahoo.com/tools

[Ladislav Vobr]

Post by lauren
I am assuming you work for an ISP... I would bet that one
of your customers was attempting to do something nasty on
a US Air Force system and tripped one or more of the IDSs.
Therefore your entire block of IPs probably got blocked
at the AF perimeter.

yes, these things happen to isps on daily bases, and an isp has to deal
with them, I was trying to say, that bind could log such problems, when
servers are being retried again for hours, days, weeks, and instead of
wasting the resources on them, it might optionally stop doing it.

and my another point was why bind doesn't provide the data from the
cache, when it has them as a glue from parent to the recursive clients,
but only to non-recursive ones?

and another point is why the information about ip address of the server
the data came from has disappeared from the named_cache.db of bind9.2.3,
as I see from the DNS &Bind Book, 4th-edition 9.1.0 still had it.

Ladislav

[Jim Reid]

Ladislav> I don't expect many this time, since as I have noticed isc
Ladislav> support became commercial, and post by person directly
Ladislav> from isc team in this 'still free mailing list' has
Ladislav> become so rare since that time....

Well what would you prefer Mark Andrews to do, answer questions here
when others are already answering them or work on the BIND code? Which
of these two things is the more important? BTW, ISC has been offering
commercial support for some time now and there has been no change to
the frequency of ISC postings here since then. There are probably
other reasons why Mark has been quieter than usual recently. For
example, there has recently been a bunch of new BIND releases and
there's a BIND9.3 in beta right now.....

Ladislav> will this list be considered now for "general help desk
Ladislav> questions" ?

You can consider this list to be whatever you like. The guarantees on
any information and response times in the list are worth what you're
paying for them. If you have a mission-critical DNS infrastructure --
and who doesn't? -- perhaps you should have a support contract that
reflects this instead of relying on the goodwill of the BIND user
mailing lists?

Ladislav> I personally think bind is great product and isc.org
Ladislav> great company, but feeling sad from the selective
Ladislav> approach perhaps isc is going to acquire now, about what
Ladislav> should be answered for free and what is going to be "3rd
Ladislav> line support" and people in the mailing list will never
Ladislav> see it, correct me if I am wrong

ISC is not a commercial software company. It gives its software away
for free. ISC even permits others to build commercial products on top
of it for free. ISC relies on donations to pay its staff, provide
computers and bandwidth for downloads, get electricity in its offices,
etc, etc. The programmers who bring us BIND have families to support
and bills to pay. Offering commercial support is (a) a service many
people want to buy; (b) a way of diversifying and extending ISC's
funding base. Both of these have to be Very Good Things.

Managements like to know all their hardware and software is covered by
support contracts, if only for the peace of mind. Anyone who can pay
for a BIND support contract should do that. Even if they don't really
need the support. What they'd then be doing is helping to sustain BIND
development by funding ISC. That's in everyone's best interests.

[Ladislav Vobr]

dear jim,

thanks a lot for your recommendation on my subject, which was "BIND
BOTTLENECK: internal 90 seconds query time-out & recursive-clients
limit" it is quite clear to me, how to handle it :-)

in case of this reply of yours, what you said is true

Post by Jim Reid
any information and response times in the list are worth what you're
paying for them.

Ladislav

[Jim Reid]

Ladislav> when you ask for ANY with a client with rd flag it
Ladislav> doesn't provide a records with cridibilityy level
Ladislav> 'glue', it returnsI believe only 'answer' and
Ladislav> 'authanswer' credibility,

No it doesn't: whatever "it" is. Maybe some other DNS implementation
might do something like that, but BIND clearly doesn't. The output
from the OP's dig postings showed that. As does the output for
ladislav.name.ae below.

BTW, the setting of header bits other than aa -- authoritative answer
-- are irrelevant to this discussion. All the rd bit does is tell the
server that's being queried that the client would like the server to
do recursion for them. Whether the server does that or not is a policy
decision. Most TLD servers have recursion disabled for obvious
reasons. [In fact no authoritative server should offer recursive
service, but that's a story for another time.] So when anything sets
the rd bit and queries the .com name servers (say), all they'll get
back is a referral to the delegated zone's name servers because the
.com name servers won't resolve www.google.com or whatever.

Ladislav> my point is how come the glue
Ladislav> from the parent is considered as a 'answer' credibility
Ladislav> in the mentioned case.

An answer is an answer. BIND's credibility ranking determines whether
certain responses are more trustworthy than others. For instance, the
Answer Section of an authoritative reply from an authoritative name
server is more believable than something in the Additional Section of
non-authoritative answer. This credibility ranking determines what
data goes in BIND's cache and how/when it gets replaced.

Data of low credibility can be cached -- after all it's better than
nothing -- but may well get displaced later by more credible data as a
name server comes across "better data" during routine operations. For
instance, it'll cache the referral info from a TLD: say the NS and any
glue records for google.com returned by the .com servers. This
response from the .com name servers is of course non-authoritative
because they don't serve google.com. But if the local name server then
looks up www.google.com, it'll query one of google.com's servers. This
should provide an answer that also includes definitive, authoritative
data about google.com's name servers and their addresses. That data
will be more credible than the referral data that was stored earlier.
So the old cached data gets displaced by the newer, more trustworthy
info from the google.com servers.

A QTYPE of ANY means "give me any records you have for this name".
That's all. So whatever is in the server's cache for that name gets
returned. This might or might not be all the RRs that exist for the
name. The response might or might not be authoritative. The data could
even be of different credibility weighting: a name's MX records might
have came from the Answer Section of an authoritative reply, but an A
or AAAA record for that name came from a referral or the Additional
Section of some other reply.

You seem to be under the impression that BIND's credibility weighting
to response data determines how it will resolve some query. It
doesn't. The server doesn't say to itself "I've only cached lowest
credibility data for this incoming query. Let's go and hunt for
definitive data from the zone's authoritatve servers and return that
to the client." The server answers from its cache if that has data
that will answer the query: credibility weighting doesn't come into
it. Otherwise it resolves the name, caches the responses during
resolving before returning an answer.

Ladislav> I have a sample domain ladislav.name.ae it has 5 fake
Ladislav> unreachable nameservers, there is no way you can get
Ladislav> them with rd flag using ANY or NS type, since they are
Ladislav> in the cache only as a glue.

So what?

Here's what I get when I lookup this domain:

% dig ladislav.name.ae any

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 59905
;; flags: qr rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 5, ADDITIONAL: 5

;; QUESTION SECTION:
;ladislav.name.ae. IN ANY

;; ANSWER SECTION:
ladislav.name.ae. 10800 IN NS fake5.ladislav.name.ae.
ladislav.name.ae. 10800 IN NS fake1.ladislav.name.ae.
ladislav.name.ae. 10800 IN NS fake2.ladislav.name.ae.
ladislav.name.ae. 10800 IN NS fake3.ladislav.name.ae.
ladislav.name.ae. 10800 IN NS fake4.ladislav.name.ae.

;; AUTHORITY SECTION:
ladislav.name.ae. 10800 IN NS fake3.ladislav.name.ae.
ladislav.name.ae. 10800 IN NS fake4.ladislav.name.ae.
ladislav.name.ae. 10800 IN NS fake5.ladislav.name.ae.
ladislav.name.ae. 10800 IN NS fake1.ladislav.name.ae.
ladislav.name.ae. 10800 IN NS fake2.ladislav.name.ae.

;; ADDITIONAL SECTION:
fake1.ladislav.name.ae. 10800 IN A 10.1.1.1
fake2.ladislav.name.ae. 10800 IN A 10.2.2.2
fake3.ladislav.name.ae. 10800 IN A 10.3.3.3
fake4.ladislav.name.ae. 10800 IN A 10.4.4.4
fake5.ladislav.name.ae. 10800 IN A 10.5.5.5

;; Query time: 2596 msec

Note the long query time. The Answer Section contains your zone's NS
records and the A records for the glue is in the Additional Section. This
is how things are expected to be. So no surprises there then. Note too
that my server didn't try to query your servers for more info about
ladislav.name.ae. The referral was enough to populate my server's
cache with data, albeit not the most credible data, and so answer the
query I made with dig.

% dig fake1.ladislav.name.ae any

;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39902
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 5, ADDITIONAL: 4

;; QUESTION SECTION:
;fake1.ladislav.name.ae. IN ANY

;; ANSWER SECTION:
fake1.ladislav.name.ae. 10684 IN A 10.1.1.1

;; AUTHORITY SECTION:
ladislav.name.ae. 10684 IN NS fake3.ladislav.name.ae.
ladislav.name.ae. 10684 IN NS fake4.ladislav.name.ae.
ladislav.name.ae. 10684 IN NS fake5.ladislav.name.ae.
ladislav.name.ae. 10684 IN NS fake1.ladislav.name.ae.
ladislav.name.ae. 10684 IN NS fake2.ladislav.name.ae.

;; ADDITIONAL SECTION:
fake2.ladislav.name.ae. 10684 IN A 10.2.2.2
fake3.ladislav.name.ae. 10684 IN A 10.3.3.3
fake4.ladislav.name.ae. 10684 IN A 10.4.4.4
fake5.ladislav.name.ae. 10684 IN A 10.5.5.5

;; Query time: 12 msec

Note the query time now. This is a local lookup. There's no attempt to
query these bogus name servers of yours. My server is answering
non-authoritatively from its cache. You can also see that from the
lower TTLs. My server is returning the glue record for
fake1.ladislav.name.ae that it cached from the previous lookup.

Ladislav> This seemed confusing to me, how come the answer from
Ladislav> the parent is sometimes considered answer and sometimes
Ladislav> as a glue?

An answer that you call "glue" -- it's actually a referral -- is still
an answer.

Ladislav> Does it depend on the rr type is NS records a different ?

Of course not. There is nothing magical about queries or answers for
NS records. In fact, name servers almost never query for NS records
explicitly. They learn about the NS records and the corresponding A or
AAAA records for some zone in the course of resolving.

[Ladislav Vobr]

Dear jim,

if you look back in the list, I have raised this issue here several
times, and I got couple of answers, which made me believe recursive
server will not provide a data cached as glue to a recursive clients,
and I am really facing this kind of problem, if you have a minute you
can look at

http://marc.theaimsgroup.com/?l=bind-users&m=107826242922419&w=2
http://marc.theaimsgroup.com/?l=bind-users&m=107826306124991&w=2

my comments below

Post by Jim Reid
Ladislav> when you ask for ANY with a client with rd flag it
Ladislav> doesn't provide a records with cridibilityy level
Ladislav> 'glue', it returnsI believe only 'answer' and
Ladislav> 'authanswer' credibility,
No it doesn't: whatever "it" is. Maybe some other DNS implementation
might do something like that, but BIND clearly doesn't. The output
from the OP's dig postings showed that. As does the output for
ladislav.name.ae below.
BTW, the setting of header bits other than aa -- authoritative answer
-- are irrelevant to this discussion. All the rd bit does is tell the
server that's being queried that the client would like the server to
do recursion for them. Whether the server does that or not is a policy
decision. Most TLD servers have recursion disabled for obvious
reasons. [In fact no authoritative server should offer recursive
service, but that's a story for another time.] So when anything sets
the rd bit and queries the .com name servers (say), all they'll get
back is a referral to the delegated zone's name servers because the
.com name servers won't resolve www.google.com or whatever.

yes agreed, and in the initial example, and in my case as well, server
provides a recursion, which is clear from the RA flag in the response.
My point was if the recursion is available, how come it doesn't even
check with the authoritative servers, and just returns what I think is a
glue from parent, since the cache was empty before

Post by Jim Reid
Ladislav> my point is how come the glue
Ladislav> from the parent is considered as a 'answer' credibility
Ladislav> in the mentioned case.
An answer is an answer. BIND's credibility ranking determines whether
certain responses are more trustworthy than others. For instance, the
Answer Section of an authoritative reply from an authoritative name
server is more believable than something in the Additional Section of
non-authoritative answer. This credibility ranking determines what
data goes in BIND's cache and how/when it gets replaced.
Data of low credibility can be cached -- after all it's better than
nothing -- but may well get displaced later by more credible data as a
name server comes across "better data" during routine operations. For
instance, it'll cache the referral info from a TLD: say the NS and any
glue records for google.com returned by the .com servers. This
response from the .com name servers is of course non-authoritative
because they don't serve google.com. But if the local name server then
looks up www.google.com, it'll query one of google.com's servers. This
should provide an answer that also includes definitive, authoritative
data about google.com's name servers and their addresses. That data
will be more credible than the referral data that was stored earlier.
So the old cached data gets displaced by the newer, more trustworthy
info from the google.com servers.

agreed, there are different credibilities obtained different ways, but
still it doesn't explain, why when the recursion is available and
required it doesn't even bother to check with authoritative servers,
seems surprising to me, also because from my side I have different
experince with ladislav.name.ae but as well as other domains, I just did
this domain as a test for trying to understand bind behaviour in case
all nameservers are unreachable

Post by Jim Reid
A QTYPE of ANY means "give me any records you have for this name".
That's all. So whatever is in the server's cache for that name gets
returned.

well i would expect this to be definitely true for +norec flag, or if
the recursion is not available, shouldn't this kind of behaviour depends
on recursion required flag and recursion available, shouldn't
caching-only server try to do it's best, if it supports recursion and it
is required, to get better answer than a glue/referral?

Post by Jim Reid
This might or might not be all the RRs that exist for the
name. The response might or might not be authoritative. The data could
even be of different credibility weighting: a name's MX records might
have came from the Answer Section of an authoritative reply, but an A
or AAAA record for that name came from a referral or the Additional
Section of some other reply.
You seem to be under the impression that BIND's credibility weighting
to response data determines how it will resolve some query. It
doesn't. The server doesn't say to itself "I've only cached lowest
credibility data for this incoming query. Let's go and hunt for
definitive data from the zone's authoritatve servers and return that
to the client." The server answers from its cache if that has data
that will answer the query: credibility weighting doesn't come into
it. Otherwise it resolves the name, caches the responses during
resolving before returning an answer.

I thought there are certain situations, when bind triggers this kind of
behaviour. Seems logical to me that when I am recursive client unable to
lookup the full answer myself,and my chaching server provides recursion,
I don't expect a glue/referral to be the answer, when the authoritative
servers are up and running and reachable? Perhaps my expectations are
too high :-), in this kind of logic, why does it bother going to .com
servers, and giving me ericsson.com refferal, it can perhaps give me as
well refferal only from . root servers, and tells me that's all I have
my dear recursive client

Post by Jim Reid
Ladislav> I have a sample domain ladislav.name.ae it has 5 fake
Ladislav> unreachable nameservers, there is no way you can get
Ladislav> them with rd flag using ANY or NS type, since they are
Ladislav> in the cache only as a glue.
So what?

could this be bind specific, since in my case I have 9.2.3 as well as
8.3.4 and 9.3.0beta4 but I am unable to get response like you, instead
of it, I see traffic to all fake nameservers, do you have any hints what
might be causing it, I have experince it several times, that certain
parts of the cache, recursive clients can not see. As you can see from
my previous example, I ran snoop and after issuing the dig with rd flag,
it keeps retrying to each ns records, while with +norec it answers :-(.
This looks very weird to me, I tried even on several nameservers and
it's the same, it timesout although it provides it with +norec.

# dig any ladislav.name.ae @194.170.1.6 +time=20

; <<>> DiG 9.3.0beta3 <<>> any ladislav.name.ae @194.170.1.6 +time=20
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 764
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;ladislav.name.ae. IN ANY

;; Query time: 10006 msec
;; SERVER: 194.170.1.6#53(194.170.1.6)
;; WHEN: Sun Jun 13 08:13:19 2004
;; MSG SIZE rcvd: 34


# dig any ladislav.name.ae @194.170.1.6 +norec

; <<>> DiG 9.3.0beta3 <<>> any ladislav.name.ae @194.170.1.6 +norec
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1770
;; flags: qr ra; QUERY: 1, ANSWER: 0, AUTHORITY: 5, ADDITIONAL: 0

;; QUESTION SECTION:
;ladislav.name.ae. IN ANY

;; AUTHORITY SECTION:
ladislav.name.ae. 10653 IN NS fake1.ladislav.name.ae.
ladislav.name.ae. 10653 IN NS fake2.ladislav.name.ae.
ladislav.name.ae. 10653 IN NS fake3.ladislav.name.ae.
ladislav.name.ae. 10653 IN NS fake4.ladislav.name.ae.
ladislav.name.ae. 10653 IN NS fake5.ladislav.name.ae.

;; Query time: 2 msec
;; SERVER: 194.170.1.6#53(194.170.1.6)
;; WHEN: Sun Jun 13 08:11:41 2004
;; MSG SIZE rcvd: 134

# dig any ericsson.com @194.170.1.6

; <<>> DiG 9.3.0beta3 <<>> any ericsson.com @194.170.1.6
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1107
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 2, ADDITIONAL: 2

;; QUESTION SECTION:
;ericsson.com. IN ANY

;; ANSWER SECTION:
ericsson.com. 172800 IN NS ns1.euro909.com.
ericsson.com. 172800 IN NS ns3.euro909.com.

;; AUTHORITY SECTION:
ericsson.com. 172800 IN NS ns1.euro909.com.
ericsson.com. 172800 IN NS ns3.euro909.com.

;; ADDITIONAL SECTION:
ns1.euro909.com. 46213 IN A 212.209.52.2
ns3.euro909.com. 42075 IN A 194.52.6.2

;; Query time: 228 msec
;; SERVER: 194.170.1.6#53(194.170.1.6)
;; WHEN: Sun Jun 13 08:11:55 2004
;; MSG SIZE rcvd: 134

snap from my named_dump.db
; glue
ladislav.name.ae. 10473 NS fake1.ladislav.name.ae.
10473 NS fake2.ladislav.name.ae.
10473 NS fake3.ladislav.name.ae.
10473 NS fake4.ladislav.name.ae.
10473 NS fake5.ladislav.name.ae.
; glue
fake1.ladislav.name.ae. 10473 A 10.1.1.1
; glue
fake2.ladislav.name.ae. 10473 A 10.2.2.2
; glue
fake3.ladislav.name.ae. 10473 A 10.3.3.3
; glue
fake4.ladislav.name.ae. 10473 A 10.4.4.4
; glue
fake5.ladislav.name.ae. 10473 A 10.5.5.5

Post by Jim Reid
% dig ladislav.name.ae any
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 59905
;; flags: qr rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 5, ADDITIONAL: 5
;ladislav.name.ae. IN ANY
ladislav.name.ae. 10800 IN NS fake5.ladislav.name.ae.
ladislav.name.ae. 10800 IN NS fake1.ladislav.name.ae.
ladislav.name.ae. 10800 IN NS fake2.ladislav.name.ae.
ladislav.name.ae. 10800 IN NS fake3.ladislav.name.ae.
ladislav.name.ae. 10800 IN NS fake4.ladislav.name.ae.
ladislav.name.ae. 10800 IN NS fake3.ladislav.name.ae.
ladislav.name.ae. 10800 IN NS fake4.ladislav.name.ae.
ladislav.name.ae. 10800 IN NS fake5.ladislav.name.ae.
ladislav.name.ae. 10800 IN NS fake1.ladislav.name.ae.
ladislav.name.ae. 10800 IN NS fake2.ladislav.name.ae.
fake1.ladislav.name.ae. 10800 IN A 10.1.1.1
fake2.ladislav.name.ae. 10800 IN A 10.2.2.2
fake3.ladislav.name.ae. 10800 IN A 10.3.3.3
fake4.ladislav.name.ae. 10800 IN A 10.4.4.4
fake5.ladislav.name.ae. 10800 IN A 10.5.5.5
;; Query time: 2596 msec
Note the long query time. The Answer Section contains your zone's NS
records and the A records for the glue is in the Additional Section. This
is how things are expected to be. So no surprises there then. Note too
that my server didn't try to query your servers for more info about
ladislav.name.ae. The referral was enough to populate my server's
cache with data, albeit not the most credible data, and so answer the
query I made with dig.
% dig fake1.ladislav.name.ae any
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39902
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 5, ADDITIONAL: 4
;fake1.ladislav.name.ae. IN ANY
fake1.ladislav.name.ae. 10684 IN A 10.1.1.1
ladislav.name.ae. 10684 IN NS fake3.ladislav.name.ae.
ladislav.name.ae. 10684 IN NS fake4.ladislav.name.ae.
ladislav.name.ae. 10684 IN NS fake5.ladislav.name.ae.
ladislav.name.ae. 10684 IN NS fake1.ladislav.name.ae.
ladislav.name.ae. 10684 IN NS fake2.ladislav.name.ae.
fake2.ladislav.name.ae. 10684 IN A 10.2.2.2
fake3.ladislav.name.ae. 10684 IN A 10.3.3.3
fake4.ladislav.name.ae. 10684 IN A 10.4.4.4
fake5.ladislav.name.ae. 10684 IN A 10.5.5.5
;; Query time: 12 msec
Note the query time now. This is a local lookup. There's no attempt to
query these bogus name servers of yours. My server is answering
non-authoritatively from its cache. You can also see that from the
lower TTLs. My server is returning the glue record for
fake1.ladislav.name.ae that it cached from the previous lookup.
Ladislav> This seemed confusing to me, how come the answer from
Ladislav> the parent is sometimes considered answer and sometimes
Ladislav> as a glue?
An answer that you call "glue" -- it's actually a referral -- is still
an answer.
Ladislav> Does it depend on the rr type is NS records a different ?
Of course not. There is nothing magical about queries or answers for
NS records. In fact, name servers almost never query for NS records
explicitly. They learn about the NS records and the corresponding A or
AAAA records for some zone in the course of resolving.

Thank you jim, do you have any explanation for the behaivour I am facing

Ladislav

[Ladislav Vobr]

Post by Jim Reid
You seem to be under the impression that BIND's credibility weighting
to response data determines how it will resolve some query. It
doesn't. The server doesn't say to itself "I've only cached lowest
credibility data for this incoming query. Let's go and hunt for
definitive data from the zone's authoritatve servers and return that
to the client." The server answers from its cache if that has data
that will answer the query: credibility weighting doesn't come into
it. Otherwise it resolves the name, caches the responses during
resolving before returning an answer.

It does, jim, and I have to say it, your reply is incorrect, bind 8.3.4
and 9.3.0beta4 and 9.2.3 all of these are reluctant to provide the
answers to recursive client if it cached with credibility *glue*.

I have done some troubleshooting, and seems that there are discrepancies
amongst different bind servers how they treat non-authoritative records
let's call it glue, some like 8.3.4 returns these in the ANSWER section,
and caching servers cache it under the credibility tag *answer* and
without a hesitation provide it to a recursive client, others like 8.4.1
and 9.2.3 or 9.3.0beta4 returns this in the authority section and it is
cached by the recursive server with credibility level *glue*, which is
in no way provided to the recursive clients.

This credibility levels and their impact is very poorly documented, and
not even discussed, should it be clear after so many years of bind
servicing the Internet community, till today there is no paper about how
it effect the recursive and non-recursive clients, how to troubleshoot,
and even here in the mailing list one can not get a clear idea, how this
was designed, what it is suppose to do, I think bind is great product,
but shouldn't this be published so everybody knows how to
troubleshoot:-( it is very simple once you figure it out.

BTW: the reason you got the reply, is imho that we have set up here
anycast clusters and you got the answer for ANY query from 8.3.4 which
gives it in the *answer* section, not the same I am getting from inside,
where is different anycast cluster serves by 9.2.3 and 8.4.1, thus I am
always getting it in the authority section and cache it as a *GLUE*,
WHICH IS NOT PROVIDED TO RECURSION DESIRED CLIENTS, BUT INSTEAD OF IT
BIND TRIGGERS A QUERY(S) (can be hundereds of queries) TO AUTH SERVERS.

P.S.: so to explain the case of ericsson.com correctly, if the gtld .com
server returs the data for ANY query in the authority section, and not
in the answer section, the caching server will be forced to cached as a
*glue* only, thus will be forced to follow up up to the end with the
authoritative server for to provide the recursive client with the
*answer* or better *authanswer*

Ladislav

[Jim Reid]

Ladislav> what's so special about ANY?

Nothing. You just don't seem to understand what it means. A QYTPE of
ANY means "give me whatever RRs you have for this name". That's all.
See my earlier posting for more info.

Ladislav> Why any recursive servers don't do it's best to satisfy
Ladislav> the recursive client with the reply from the authoritative
Ladislav> server, that's why we call it recursive right?

Wrong. We call it recursive because the server is able to recursively
make iterative queries to resolve a query on behalf of some client.
It doesn't mean the server does that: it can answer from its cache
which might or might not have been populated with data returned from
earlier queries to authoritative servers. No assumptions can or should
be made about how a recursive server provides answers. It should of
course interrogate authoritative servers when nothing has been
cached. But that cannot be guaranteed. And even if it does query
authoritative servers, the answer might not be correct. The DNS is
loosely coupled remember. It can take time for a zone's authoritative
servers to converge on the same copy of the zone data after the zone
gets updated. They don't all update the zone simultaneously.

You seem to think that an ANY QTYPE means a server must retrieve every
RR for the name. That's not the case. In fact this is impossible. The
master server could change the RRs immediately after answering your
hypothetical EVERY query before that reply gets back to the client.
It's not even the case that a server must query an authoritative
server in order to respond to an ANY query.

Remember too that one of the key strengths of the DNS is caching. In
some sense this means that recursive servers are lazy. They'll answer
from cache every time unless there's nothing relevant in the cache and
they're forced to resolve something. This is why people need to think
carefully about TTL values. How many times have we seen postings here
where there's been a long-lived TTL for a web or mail server that then
gets renumbered and the poster whines that traffic still goes to the
old address even though they've updated the zone?

Ladislav> to do this kind of work for the
Ladislav> client, how can it take answer from the parent and
Ladislav> consider the task done?

Because that's how the DNS works.

Ladislav> I have problem with ladislav.name.ae

.... snipped ....

This appears to be either a wierd local set-up or else you have a
misunderstanding of what's going on.

[Ladislav Vobr]

jim, thanks for your support, unfortunately I have to go, I will go
through you mail today evening,

I am really having problems, having bind retrying to authoritative
servers, I am surprised that dig work for you, since it doesn't work for
me and it tries for several time all the authoritative servers I have
and than it times-out:-( I hope it might be configuration issue, but I
doubt it.

I was really under impression, cache type *glue* is not provided to a
recursive clients, that's how it works here for me:-(

just a sample

ns3.emirates.net.ae# jobs
[1] + Running snoop 10.1.1.1
ns3.emirates.net.ae#
ns3.emirates.net.ae#
ns3.emirates.net.ae# dig any ladislav.name.ae
ns3.emirates.net.ae -> 10.1.1.1 DNS C ladislav.name.ae. Internet * ?
ns3.emirates.net.ae -> 10.1.1.1 DNS C ladislav.name.ae. Internet * ?
ns3.emirates.net.ae -> 10.1.1.1 DNS C fake1.ladislav.name.ae.
Internet Unknown (38) ?
ns3.emirates.net.ae -> 10.1.1.1 DNS C fake2.ladislav.name.ae.
Internet Unknown (38) ?
ns3.emirates.net.ae -> 10.1.1.1 DNS C fake3.ladislav.name.ae.
Internet Unknown (38) ?
ns3.emirates.net.ae -> 10.1.1.1 DNS C fake4.ladislav.name.ae.
Internet Unknown (38) ?
ns3.emirates.net.ae -> 10.1.1.1 DNS C fake5.ladislav.name.ae.
Internet Unknown (38) ?

; <<>> DiG 9.2.3 <<>> any ladislav.name.ae
;; global options: printcmd
;; connection timed out; no servers could be reached

ns3.emirates.net.ae# dig any ladislav.name.ae +norec

; <<>> DiG 9.2.3 <<>> any ladislav.name.ae +norec
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 47234
;; flags: qr ra; QUERY: 1, ANSWER: 0, AUTHORITY: 5, ADDITIONAL: 0

;; QUESTION SECTION:
;ladislav.name.ae. IN ANY

;; AUTHORITY SECTION:
ladislav.name.ae. 3278 IN NS fake2.ladislav.name.ae.
ladislav.name.ae. 3278 IN NS fake3.ladislav.name.ae.
ladislav.name.ae. 3278 IN NS fake4.ladislav.name.ae.
ladislav.name.ae. 3278 IN NS fake5.ladislav.name.ae.
ladislav.name.ae. 3278 IN NS fake1.ladislav.name.ae.

;; Query time: 42 msec
;; SERVER: 194.170.1.99#53(194.170.1.99)
;; WHEN: Sat Jun 12 15:24:18 2004
;; MSG SIZE rcvd: 134

can you explain this ?

Ladislav

Post by Jim Reid
Ladislav> what's so special about ANY?
Nothing. You just don't seem to understand what it means. A QYTPE of
ANY means "give me whatever RRs you have for this name". That's all.
See my earlier posting for more info.
Ladislav> Why any recursive servers don't do it's best to satisfy
Ladislav> the recursive client with the reply from the authoritative
Ladislav> server, that's why we call it recursive right?
Wrong. We call it recursive because the server is able to recursively
make iterative queries to resolve a query on behalf of some client.
It doesn't mean the server does that: it can answer from its cache
which might or might not have been populated with data returned from
earlier queries to authoritative servers. No assumptions can or should
be made about how a recursive server provides answers. It should of
course interrogate authoritative servers when nothing has been
cached. But that cannot be guaranteed. And even if it does query
authoritative servers, the answer might not be correct. The DNS is
loosely coupled remember. It can take time for a zone's authoritative
servers to converge on the same copy of the zone data after the zone
gets updated. They don't all update the zone simultaneously.
You seem to think that an ANY QTYPE means a server must retrieve every
RR for the name. That's not the case. In fact this is impossible. The
master server could change the RRs immediately after answering your
hypothetical EVERY query before that reply gets back to the client.
It's not even the case that a server must query an authoritative
server in order to respond to an ANY query.
Remember too that one of the key strengths of the DNS is caching. In
some sense this means that recursive servers are lazy. They'll answer
from cache every time unless there's nothing relevant in the cache and
they're forced to resolve something. This is why people need to think
carefully about TTL values. How many times have we seen postings here
where there's been a long-lived TTL for a web or mail server that then
gets renumbered and the poster whines that traffic still goes to the
old address even though they've updated the zone?
Ladislav> to do this kind of work for the
Ladislav> client, how can it take answer from the parent and
Ladislav> consider the task done?
Because that's how the DNS works.
Ladislav> I have problem with ladislav.name.ae
.... snipped ....
This appears to be either a wierd local set-up or else you have a
misunderstanding of what's going on.

[Barry Margolin]

In article <caepoh$31cp$1 at sf1.isc.org>,

Post by Ladislav Vobr
jim, thanks for your support, unfortunately I have to go, I will go
through you mail today evening,
I am really having problems, having bind retrying to authoritative
servers, I am surprised that dig work for you, since it doesn't work for
me and it tries for several time all the authoritative servers I have
and than it times-out:-( I hope it might be configuration issue, but I
doubt it.
I was really under impression, cache type *glue* is not provided to a
recursive clients, that's how it works here for me:-(
just a sample
ns3.emirates.net.ae# jobs
[1] + Running snoop 10.1.1.1
ns3.emirates.net.ae#
ns3.emirates.net.ae#
ns3.emirates.net.ae# dig any ladislav.name.ae
ns3.emirates.net.ae -> 10.1.1.1 DNS C ladislav.name.ae. Internet * ?
ns3.emirates.net.ae -> 10.1.1.1 DNS C ladislav.name.ae. Internet * ?
ns3.emirates.net.ae -> 10.1.1.1 DNS C fake1.ladislav.name.ae.
Internet Unknown (38) ?
ns3.emirates.net.ae -> 10.1.1.1 DNS C fake2.ladislav.name.ae.
Internet Unknown (38) ?
ns3.emirates.net.ae -> 10.1.1.1 DNS C fake3.ladislav.name.ae.
Internet Unknown (38) ?
ns3.emirates.net.ae -> 10.1.1.1 DNS C fake4.ladislav.name.ae.
Internet Unknown (38) ?
ns3.emirates.net.ae -> 10.1.1.1 DNS C fake5.ladislav.name.ae.
Internet Unknown (38) ?
; <<>> DiG 9.2.3 <<>> any ladislav.name.ae
;; global options: printcmd
;; connection timed out; no servers could be reached
ns3.emirates.net.ae# dig any ladislav.name.ae +norec
; <<>> DiG 9.2.3 <<>> any ladislav.name.ae +norec
;; global options: printcmd
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 47234
;; flags: qr ra; QUERY: 1, ANSWER: 0, AUTHORITY: 5, ADDITIONAL: 0
;ladislav.name.ae. IN ANY
ladislav.name.ae. 3278 IN NS fake2.ladislav.name.ae.
ladislav.name.ae. 3278 IN NS fake3.ladislav.name.ae.
ladislav.name.ae. 3278 IN NS fake4.ladislav.name.ae.
ladislav.name.ae. 3278 IN NS fake5.ladislav.name.ae.
ladislav.name.ae. 3278 IN NS fake1.ladislav.name.ae.
;; Query time: 42 msec
;; SERVER: 194.170.1.99#53(194.170.1.99)
;; WHEN: Sat Jun 12 15:24:18 2004
;; MSG SIZE rcvd: 134
can you explain this ?

10.x.x.x addresses are private addresses that are not reachable from the
Internet. Why do you have your domain delegated to these unusable
addresses?

[Ladislav Vobr]

Post by Barry Margolin

Post by Ladislav Vobr
; <<>> DiG 9.2.3 <<>> any ladislav.name.ae
;; global options: printcmd
;; connection timed out; no servers could be reached
ns3.emirates.net.ae# dig any ladislav.name.ae +norec
; <<>> DiG 9.2.3 <<>> any ladislav.name.ae +norec
;; global options: printcmd
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 47234
;; flags: qr ra; QUERY: 1, ANSWER: 0, AUTHORITY: 5, ADDITIONAL: 0
;ladislav.name.ae. IN ANY
ladislav.name.ae. 3278 IN NS fake2.ladislav.name.ae.
ladislav.name.ae. 3278 IN NS fake3.ladislav.name.ae.
ladislav.name.ae. 3278 IN NS fake4.ladislav.name.ae.
ladislav.name.ae. 3278 IN NS fake5.ladislav.name.ae.
ladislav.name.ae. 3278 IN NS fake1.ladislav.name.ae.
;; Query time: 42 msec
;; SERVER: 194.170.1.99#53(194.170.1.99)
;; WHEN: Sat Jun 12 15:24:18 2004
;; MSG SIZE rcvd: 134
can you explain this ?

10.x.x.x addresses are private addresses that are not reachable from the
Internet. Why do you have your domain delegated to these unusable
addresses?

I set this up purposely, since I faced some weird behaviour, and also I
was wondering how bind handles the situation when all servers are
unreachable,

I have two issues with this setup, as my old posts,

1. bind sends around 150 packets total to all nameservers, when I
request single name for any host from this domain, which seems quite a
lot for me, and it seems to be doing it for every new name again, looks
like pretty ddos amplifier I am trying to understand this logic.

2. I have this +norec/rd flag issue, when I am unable to see the *glue*
from the cache with rd flag.

Ladislav

[Kevin Darcy]

Post by Jim Reid
Ladislav> what's so special about ANY?
Nothing. You just don't seem to understand what it means. A QYTPE of
ANY means "give me whatever RRs you have for this name". That's all.
See my earlier posting for more info.
Ladislav> Why any recursive servers don't do it's best to satisfy
Ladislav> the recursive client with the reply from the authoritative
Ladislav> server, that's why we call it recursive right?
Wrong. We call it recursive because the server is able to recursively
make iterative queries to resolve a query on behalf of some client.
It doesn't mean the server does that: it can answer from its cache
which might or might not have been populated with data returned from
earlier queries to authoritative servers. No assumptions can or should
be made about how a recursive server provides answers. It should of
course interrogate authoritative servers when nothing has been
cached. But that cannot be guaranteed. And even if it does query
authoritative servers, the answer might not be correct. The DNS is
loosely coupled remember. It can take time for a zone's authoritative
servers to converge on the same copy of the zone data after the zone
gets updated. They don't all update the zone simultaneously.
You seem to think that an ANY QTYPE means a server must retrieve every
RR for the name. That's not the case. In fact this is impossible. The
master server could change the RRs immediately after answering your
hypothetical EVERY query before that reply gets back to the client.
It's not even the case that a server must query an authoritative
server in order to respond to an ANY query.
Remember too that one of the key strengths of the DNS is caching. In
some sense this means that recursive servers are lazy. They'll answer
from cache every time unless there's nothing relevant in the cache and
they're forced to resolve something. This is why people need to think
carefully about TTL values. How many times have we seen postings here
where there's been a long-lived TTL for a web or mail server that then
gets renumbered and the poster whines that traffic still goes to the
old address even though they've updated the zone?

That's fine and dandy. We all understand that DNS is "loosely coupled",
and that caching requires all sorts of tradeoffs and compromises. But I
think personally QTYPE=* has been compromised to the point of almost
being unusable for its originally-intended purpose. What if instead of
defining a valid, complete QTYPE=* response from cache as "whatever you
happen to have in your cache", which as Ladislav rightly points out
makes a mockery of the RA flag, we define it as

"RRsets for all record types that were seen in the most recent
QTYPE=* response, if none of them have expired"

?? Certainly we don't expect recursive servers to recurse *every*
QTYPE=* query, on the offchance that maybe some RRset has changed, but
certainly if all of the RRsets for all of the record types previously
seen in a QTYPE=* response are still present in the cache, regardless of
whether those RRsets were from the original QTYPE=* query, or from other
queries, or combination of the two, why not synthesize a response from
them? That seems perfectly reasonable to me, and not unduly burdensome
to recursing servers. Note that if the recursing server happens to
notice that it is missing exactly *one* of the RRsets (a fairly common
occurrence, I would expect, since A records tend to have smaller TTLs
than anything else), then it could make a choice between recursing the
whole QTYPE=* query, or just fetching the missing RRset. The alternative
is for the client to make the A-record query itself, along with queries
for whatever other record types in which it is interested, so from the
recursive-server's standpoint, this can only be a win.

I can understand that this definition could raise a concern about
"record-type lock", where the recursive server might assume, in
perpetuity (or at least until the next restart/reboot), that only
certain record types exist for a given name, and thus keep synthesizing
QTYPE=* responses from the same set of record types even after one or
more types have been deleted from or added to the name. Deleted types
aren't really a problem, since according to the above definition the
answer is not valid and/or complete if any of the individual RRsets have
expired, so eventually the deleted RRset will expire and the recursive
server will be forced to fetch a new QTYPE=* answer. This "staleness"
problem is thus no worse than if the client was using type-specific
queries instead of QTYPE=*. Added types are a bit more problematic --
how can a recursive server know that records with a given type have been
added to a name, unless it's actually querying that name & type (or
QTYPE=*) from its upstream sources? This problem can be solved at the
protocol level with a very minor clarification/extension of Negative
Caching, but in the interim, some more-or-less arbitrary limit could be
set in BIND, such that a fresh QTYPE=* query is forced every so often,
in order to prevent "record-type lock". This could be just a fixed
amount of time, such as was implemented before Negative Caching was
properly specified, and like BIND 9 currently implements for "lame
server TTL"s, or it could be based on the TTLs of the various RRsets
associated with the name, either an average, the minimum, the maximum,
or the result of some sort of multivariable formula...


- Kevin

[Barry Margolin]

In article <calb87$2osn$1 at sf1.isc.org>,

Post by Kevin Darcy
That's fine and dandy. We all understand that DNS is "loosely coupled",
and that caching requires all sorts of tradeoffs and compromises. But I
think personally QTYPE=* has been compromised to the point of almost
being unusable for its originally-intended purpose.

Just what *is* that purpose? I don't see any indication in RFC 1034; no
real justification is given for its existence.

Note also that the OP has made a big deal about whether it should return
records with cred=GLUE, but the DNS specification makes no mention of
credibility levels for cached information. All it says, in section
5.3.3 (the resolver algorithm, which is used by a server when processing
a query that has RD set) is: "Step 1 searches the cache for the desired
data. If the data is in the cache, it is assumed to be good enough for
normal use."

[Kevin Darcy]

Post by Barry Margolin
In article <calb87$2osn$1 at sf1.isc.org>,

Post by Kevin Darcy
That's fine and dandy. We all understand that DNS is "loosely coupled",
and that caching requires all sorts of tradeoffs and compromises. But I
think personally QTYPE=* has been compromised to the point of almost
being unusable for its originally-intended purpose.

Just what *is* that purpose? I don't see any indication in RFC 1034; no
real justification is given for its existence.

RFCs are specification documents, they don't necessarily justify the
existence of every aspect of what they specify. But it seems rather
obvious to me that the purpose of QTYPE=* is to efficiently retrieve all
relevant RRsets owned by a particular DNS name, as opposed to querying
all of those RRsets individually. The way QTYPE=* has been implemented,
however, has rendered it so untrustworthy that very few apps that could
benefit from this efficiency even bother to issue QTYPE=* queries any
more. This is a pity, all the more so because it would be a rather
elegant way to retrieve both A and AAAA records for a given name, and
thus ease the migration to IPv6.

Post by Barry Margolin
Note also that the OP has made a big deal about whether it should return
records with cred=GLUE, but the DNS specification makes no mention of
credibility levels for cached information. All it says, in section
5.3.3 (the resolver algorithm, which is used by a server when processing
a query that has RD set) is: "Step 1 searches the cache for the desired
data. If the data is in the cache, it is assumed to be good enough for
normal use."

RFC 2181 states "... data from the additional data section, and data
from the authority section of a non-authoritative answer, should not be
cached in such a way that they would ever be returned as answers to a
received query." This applies to QTYPE=* queries just as much as any
other query type.


- Kevin

[Barry Margolin]

In article <calhbv$6ba$1 at sf1.isc.org>,

Post by Kevin Darcy

Post by Barry Margolin
In article <calb87$2osn$1 at sf1.isc.org>,

Post by Kevin Darcy
That's fine and dandy. We all understand that DNS is "loosely coupled",
and that caching requires all sorts of tradeoffs and compromises. But I
think personally QTYPE=* has been compromised to the point of almost
being unusable for its originally-intended purpose.

Just what *is* that purpose? I don't see any indication in RFC 1034; no
real justification is given for its existence.

RFCs are specification documents, they don't necessarily justify the
existence of every aspect of what they specify. But it seems rather
obvious to me that the purpose of QTYPE=* is to efficiently retrieve all
relevant RRsets owned by a particular DNS name, as opposed to querying
all of those RRsets individually. The way QTYPE=* has been implemented,
however, has rendered it so untrustworthy that very few apps that could
benefit from this efficiency even bother to issue QTYPE=* queries any
more. This is a pity, all the more so because it would be a rather
elegant way to retrieve both A and AAAA records for a given name, and
thus ease the migration to IPv6.

But RFC 1034 included an example of QTYPE=* being sent to caching
servers, showing that different servers will return different records
based on what they happened to have cached at the time. So the problem
is in the original design, not BIND's implementation.

Post by Kevin Darcy

Post by Barry Margolin
Note also that the OP has made a big deal about whether it should return
records with cred=GLUE, but the DNS specification makes no mention of
credibility levels for cached information. All it says, in section
5.3.3 (the resolver algorithm, which is used by a server when processing
a query that has RD set) is: "Step 1 searches the cache for the desired
data. If the data is in the cache, it is assumed to be good enough for
normal use."

RFC 2181 states "... data from the additional data section, and data
from the authority section of a non-authoritative answer, should not be
cached in such a way that they would ever be returned as answers to a
received query." This applies to QTYPE=* queries just as much as any
other query type.

I did notice a change related to that when we upgraded our caching
servers from BIND 8 to 9. Prior to that, if I asked for the A record of
a nameserver, I would often get the address from the glue in the parent
zone. After the upgrade, it seemed to go to the authoritative server
for this -- if all of the zone's servers were down, the query would hang
and eventually return a SERVFAIL error. The only way to get the cached
glue record was to query without the RD flag set.

However, I think ANY queries would still return whatever happened to be
in the cache, no matter how it was learned.

[Ladislav Vobr]

Post by Barry Margolin
I did notice a change related to that when we upgraded our caching
servers from BIND 8 to 9. Prior to that, if I asked for the A record of
a nameserver, I would often get the address from the glue in the parent
zone. After the upgrade, it seemed to go to the authoritative server
for this -- if all of the zone's servers were down, the query would hang
and eventually return a SERVFAIL error. The only way to get the cached
glue record was to query without the RD flag set.

barry, the change is there between 8.3.4 and 8.4.1, 8.4.1 returns is the
same way as 9 and higher, 8.3.4 returns it as a *answer*, I think this
will be very important to distinguish once it comes to dnssec. What is
glue and what is not, since the glue is not signed.

Post by Barry Margolin
However, I think ANY queries would still return whatever happened to be
in the cache, no matter how it was learned.

if it is cached with *glue* credibility it will not return it to ra
clients. This behavious as you describe is nightmare, it keeps retrying
to all nameservers if all unreachable causing incredible traffic to
remote servers and the network as well, I am sometimes seeing
nameservers querying me with 1000(one thousand)req/s with the same
request, this can really spoil lot's of things, why would ever caching
nameserver has to do such a thing, does it really help to do it this
way....?

how can we say it is perfectly fine to answer the recursive client with
non-authoritative data, when nothing was cached before this request? I
feel recursion means, if it is not available, recurse up to the source
(auth servers) and get it, not from . or 2ndlevel or 3level or 4level,
we can not stop randomly somelevel just because some binds think it was
enough steps(parent 8.3.4 thinks it is enough, parent 8.4.1 thinks try
to go to next level... it seems really very consistent:-), we always
should go up to the source *provided it was not cached before*. How will
this work in dnssec, we just answer to ra client with *glue* and tell
him be happy for it:-)?

Ladislav

[Kevin Darcy]

Post by Barry Margolin
In article <calhbv$6ba$1 at sf1.isc.org>,

Post by Kevin Darcy

Post by Barry Margolin
In article <calb87$2osn$1 at sf1.isc.org>,

Post by Kevin Darcy
That's fine and dandy. We all understand that DNS is "loosely coupled",
and that caching requires all sorts of tradeoffs and compromises. But I
think personally QTYPE=* has been compromised to the point of almost
being unusable for its originally-intended purpose.

Just what *is* that purpose? I don't see any indication in RFC 1034; no
real justification is given for its existence.

RFCs are specification documents, they don't necessarily justify the
existence of every aspect of what they specify. But it seems rather
obvious to me that the purpose of QTYPE=* is to efficiently retrieve all
relevant RRsets owned by a particular DNS name, as opposed to querying
all of those RRsets individually. The way QTYPE=* has been implemented,
however, has rendered it so untrustworthy that very few apps that could
benefit from this efficiency even bother to issue QTYPE=* queries any
more. This is a pity, all the more so because it would be a rather
elegant way to retrieve both A and AAAA records for a given name, and
thus ease the migration to IPv6.

But RFC 1034 included an example of QTYPE=* being sent to caching
servers, showing that different servers will return different records
based on what they happened to have cached at the time. So the problem
is in the original design, not BIND's implementation.

Nope. Those example queries were *non-recursive* as per the following
text in the Section 6.2 intro:

Unless otherwise noted, the queries do not have recursion desired (RD)
in the header.

Nowhere is there a specific example in RFC 1034 of a response to a
*recursive* QTYPE=* query, but one would assume, based on generic
descriptions of recursive resolvers and how they are supposed to
operate, that a recursive resolver would make its best efforts to get a
complete answer, which clearly BIND and other implementations do not.
Frankly, I think the implementors misread the RFC 1034 examples the same
way you did, and refuse to admit their mistake.


- Kevin

[Barry Margolin]

In article <caobi7$1lkv$1 at sf1.isc.org>,

Post by Kevin Darcy
Nowhere is there a specific example in RFC 1034 of a response to a
*recursive* QTYPE=* query, but one would assume, based on generic
descriptions of recursive resolvers and how they are supposed to
operate, that a recursive resolver would make its best efforts to get a
complete answer, which clearly BIND and other implementations do not.

The server algorithm says that when a query has RD set, the server
should consult its resolver. If you read the section on the resolver
algorithm, it says that if the records are already in the cache it
doesn't need to query the authoritative server.

[Ladislav Vobr]

Post by Barry Margolin
The server algorithm says that when a query has RD set, the server
should consult its resolver. If you read the section on the resolver
algorithm, it says that if the records are already in the cache it
doesn't need to query the authoritative server.

neither way it is implemented, *glue* credibility is not provided to rd
clients. When and what is cached under *glue* depends purely on *each*
bind reasoning.

Ladislav

[Kevin Darcy]

Post by Barry Margolin
In article <caobi7$1lkv$1 at sf1.isc.org>,

Post by Kevin Darcy
Nowhere is there a specific example in RFC 1034 of a response to a
*recursive* QTYPE=* query, but one would assume, based on generic
descriptions of recursive resolvers and how they are supposed to
operate, that a recursive resolver would make its best efforts to get a
complete answer, which clearly BIND and other implementations do not.

The server algorithm says that when a query has RD set, the server
should consult its resolver. If you read the section on the resolver
algorithm, it says that if the records are already in the cache it
doesn't need to query the authoritative server.

Actually, what RFC 1034 says in the resolver algorithm description is
"See if the *answer* is in local information" (emphasis mine). Not just
any old records that happen to be owned by the QNAME, but the *answer*
to the query. I suppose reasonable people could disagree over exactly
what "answer" means in that context, but it seems a rather strange
interpretation to me to say that a recursive resolver can choose to give
the client *less* of an answer than it wanted (note that the description
of the "General Lookup Function" in Section 5.2.1 refers to the client
wanting "*all* of the matching RRs" (emphasis added)), even when a
complete answer is available via recursion, recursion was requested by
the client and the availability of recursion is being advertised by the
recursive resolver via the RA bit. If the client wanted a least-efforts
answer, it wouldn't have set RD=1 in the first place, surely. The
description of the RA flag in the RFCs is not "Recursion Available,
except for QTYPE=* queries, to which I'll give partial answers from
cache if I don't feel like recursing for something better"...


- Kevin

[Guido Roeskens]

Hello,

I thought I've seen a discussion about this on namedroppers before
http://marc.theaimsgroup.com/?l=namedroppers&m=106920851901151&w=2
(start of the thread)

Post by Kevin Darcy

Post by Barry Margolin
In article <caobi7$1lkv$1 at sf1.isc.org>,

Post by Kevin Darcy
Nowhere is there a specific example in RFC 1034 of a response to a
*recursive* QTYPE=* query, but one would assume, based on generic
descriptions of recursive resolvers and how they are supposed to
operate, that a recursive resolver would make its best efforts to get a
complete answer, which clearly BIND and other implementations do not.

The server algorithm says that when a query has RD set, the server
should consult its resolver. If you read the section on the resolver
algorithm, it says that if the records are already in the cache it
doesn't need to query the authoritative server.

Actually, what RFC 1034 says in the resolver algorithm description is
"See if the *answer* is in local information" (emphasis mine). Not just
any old records that happen to be owned by the QNAME, but the *answer*
to the query. I suppose reasonable people could disagree over exactly
what "answer" means in that context, but it seems a rather strange
interpretation to me to say that a recursive resolver can choose to give
the client *less* of an answer than it wanted (note that the description
of the "General Lookup Function" in Section 5.2.1 refers to the client
wanting "*all* of the matching RRs" (emphasis added)), even when a
complete answer is available via recursion, recursion was requested by
the client and the availability of recursion is being advertised by the
recursive resolver via the RA bit. If the client wanted a least-efforts
answer, it wouldn't have set RD=1 in the first place, surely. The
description of the RA flag in the RFCs is not "Recursion Available,
except for QTYPE=* queries, to which I'll give partial answers from
cache if I don't feel like recursing for something better"...

In the thread mentioned above someone (I think it's Paul Vixie says
that QTYPE=* is some sort of debugging aid.
The problem is how a resolver should know if it has to query the
authoritative server(s) or not. It would need to keep a record of
when it last queried the authoritave server for QTYPE=* and set it's
TTL to the lowest anwer given.
Or the resolver would need to query the authoritative servers each time
for the answer, since the QTYPE=* cannot be cached.
This could overload the server easily or open doors for DOS attacks.

Guido

[Kevin Darcy]

Post by Guido Roeskens
Hello,
I thought I've seen a discussion about this on namedroppers before
http://marc.theaimsgroup.com/?l=namedroppers&m=106920851901151&w=2
(start of the thread)

Post by Kevin Darcy

Post by Barry Margolin
In article <caobi7$1lkv$1 at sf1.isc.org>,

Post by Kevin Darcy
Nowhere is there a specific example in RFC 1034 of a response to a
*recursive* QTYPE=* query, but one would assume, based on generic
descriptions of recursive resolvers and how they are supposed to
operate, that a recursive resolver would make its best efforts to get a
complete answer, which clearly BIND and other implementations do not.

The server algorithm says that when a query has RD set, the server
should consult its resolver. If you read the section on the resolver
algorithm, it says that if the records are already in the cache it
doesn't need to query the authoritative server.

Actually, what RFC 1034 says in the resolver algorithm description is
"See if the *answer* is in local information" (emphasis mine). Not just
any old records that happen to be owned by the QNAME, but the *answer*
to the query. I suppose reasonable people could disagree over exactly
what "answer" means in that context, but it seems a rather strange
interpretation to me to say that a recursive resolver can choose to give
the client *less* of an answer than it wanted (note that the description
of the "General Lookup Function" in Section 5.2.1 refers to the client
wanting "*all* of the matching RRs" (emphasis added)), even when a
complete answer is available via recursion, recursion was requested by
the client and the availability of recursion is being advertised by the
recursive resolver via the RA bit. If the client wanted a least-efforts
answer, it wouldn't have set RD=1 in the first place, surely. The
description of the RA flag in the RFCs is not "Recursion Available,
except for QTYPE=* queries, to which I'll give partial answers from
cache if I don't feel like recursing for something better"...

In the thread mentioned above someone (I think it's Paul Vixie says
that QTYPE=* is some sort of debugging aid.
The problem is how a resolver should know if it has to query the
authoritative server(s) or not. It would need to keep a record of
when it last queried the authoritave server for QTYPE=* and set it's
TTL to the lowest anwer given.
Or the resolver would need to query the authoritative servers each time
for the answer, since the QTYPE=* cannot be cached.
This could overload the server easily or open doors for DOS attacks.

I'm not suggesting that QTYPE=* answers be treated as completely
non-cacheable, nor am I suggesting that QTYPE=* responses be treated as
"atomic" cache entries that expire with the lowest-valued TTL of all
included RRsets. The resolver should be able to answer QTYPE=* queries
entirely from cache, or perhaps by fetching only 1 RRset if only 1 is
missing, as long as it can provide a "complete" answer to the client,
which it can construct from RRsets from the original query, from
subsequent queries, or some combination of the two. I guess one could
quibble about the exact definition of "complete"...

As for DoS attacks, how is it any more vulnerable than allowing clients
to make multiple type-specific queries? You could argue, I suppose, that
QTYPE=* responses are "heavier" than queries for individual record
types, and thus might fly under the radar of rate-limiters, IDS'es and
such, which only look at the number of queries. But the counter-argument
is that the non-robustness of QTYPE=* has, in reality, forced a
relaxation of those rate-limiter/IDS thresholds, and thus the position
that robust QTYPE=* treatment leads to "stealth" DoS'es is actually a
self-defeating one. Make QTYPE=* more robust, and when apps with
legitimate requirements to fetch records of multiple types are rewritten
to do so *efficiently* via QTYPE=* instead of separate queries, then you
can dial down the rate-limiter thresholds to thwart *real* DoS'es, and
everyone is happy (or at least happier).

- Kevin

[Jim Reid]

Ladislav> It does, jim, and I have to say it, your reply is
Ladislav> incorrect, bind 8.3.4 and 9.3.0beta4 and 9.2.3 all of
Ladislav> these are reluctant to provide the answers to recursive
Ladislav> client if it cached with credibility *glue*.

If that's the case, please explain why my BIND9.3 server is perfectly
happy to return the glue it got for fake1.ladislav.name.ae after it
had cached a referral for ladislav.name.ae. That was clearly shown in
my earlier posting.

Ladislav> I have done some troubleshooting, and seems that there
Ladislav> are discrepancies amongst different bind servers how
Ladislav> they treat non-authoritative records let's call it glue,

No, let's call it "non-authoritative records". Glue is a specific
technical term that has a specific meaning.

Anyway, I'm not continuing this discussion as I've got nothing to add
to what I've already said.

[Ladislav Vobr]

Post by Jim Reid
Ladislav> It does, jim, and I have to say it, your reply is
Ladislav> incorrect, bind 8.3.4 and 9.3.0beta4 and 9.2.3 all of
Ladislav> these are reluctant to provide the answers to recursive
Ladislav> client if it cached with credibility *glue*.
If that's the case, please explain why my BIND9.3 server is perfectly
happy to return the glue it got for fake1.ladislav.name.ae after it
had cached a referral for ladislav.name.ae. That was clearly shown in
my earlier posting.

Let me ask you one thing, are you sure about the credibility tag the
data got cached on your caching 9.3 server? Do you know exactly the
rules, when it is cached under which tag? And how each different bind
treats the same non-authoritative data in it's own way? And how each
caching server treats each credibility level when it comes to answer rd
clients?

I feel strange discussing things on the @isc.org mailing list, while
nobody from ISC, who sure knows these things, doesn't bother to reply
and say, yes there is a relation, and these are the rules 1,2,3...

It is simple once it is published, but till that time people live in dark.

Do you thing commercial support will explain this, or will just admit it
that there is some relation. We don't have issue paying for it, we are
already sponsoring isc hosting and operation of the f-root server in
next room, but is there any sample how does this commercial support
look, how deep does it go, and where the discussion stops? Does it
answer such a questions?

I can try go to the source code and spend nights and nights reading it,
but I just feel I can be more useful doing other things, maybe I am wrong.

P.S. the explanation is in my previous post, it is simple you cached it
from 8.3.4 thus it has *answer* credibility, but I got it from 9.2.3 or
8.4.1 or 9.3.0beta4 thus the credibility for the same thing was *glue*
and as per the unknown rule this is not provided to rd clients.

I know I have a million questions, and perhaps it looks like I am not
friend of isc, but I am, just want to understand things. Today, all
administrators look like 'not really aware', when somebody ask them, how
come it is in the cache, but bind doesn't answer with it:-( shouldn't it
answer from cache? simple question, but not simple answer....

Ladislav

[Jim Reid]

BIND has no hooks for this sort of thing. Feel free to
contribute code... Rate limiting is probably best handled by a
router or firewall in front of the name server. Perhaps you
could do that?

Ladislav> firewall will limit only total traffic or static clients
Ladislav> (you have to configure in source ip), it will not
Ladislav> dynamically limit each random customer. It means
Ladislav> basically that the service will be non-responsive for
Ladislav> all, if the total traffic is exceeded.

You obviously haven't understood what I posted. A firewall doesn't
only completely block unwanted traffic. Some firewalls *do* provide
rate limiting. As, of course, do routers.

Ladislav> The rate limiting per customer or per ip is basic thing
Ladislav> that already many applications are using, apache,
Ladislav> sendmail, sunone, iplanet... have you noticed it ?

Of course. However an HTTP or SMTP transaction is a very different
beast from a DNS transaction.

I'd also recommend that you get your customers to reconfigure
their name servers so they resolve stuff for themselves instead
of forwarding queries to your name server. That forwarding
server that sends 1200qps is anti-social and probably
broken. It might be helpful to find out why it's generating so
much traffic. Even better would be putting a stop to that much
traffic. :-)

Ladislav> Customers doing what they want, if bind can rate limit
Ladislav> them, they will ofcourse re-evaluate their behaviour,
Ladislav> because they will be forced to do it.

This is nonsense. First of all, the customers are probably not "doing
what they want". They're most likely doing what their ISP told them to
do a long time ago. Presumably neither the ISP or the customer at that
time had a clue about DNS operations and the pointless stupidity of
forwarding. [If they had, they wouldn't have configured a forwarding
setup.] The customers may well be blissfully unaware that their
forwarding name servers are pounding incessantly on the ISP's
server. The customers may well be running ancient DNS code. So they
could have name servers that don't implement negative caching asking
for the same non-existent names over and over again.

Ladislav> Since bind doesn't care about it, nobody cares, saying
Ladislav> that router will solve it? Will the router ensure that
Ladislav> *each* *random* customer will have let's say bw for
Ladislav> 20/req per second and not more, just think about it.

How someone choses to configure rate limiting on their routers is up
to them. In all likelihood, the excessive traffic will be coming from
a small number of IP addresses, so it would be trivial to make the
router rate-limit that traffic while not impeding the rest. Not that
rate limiting is the answer anyway. Applications and resolvers
sometimes do evil and stupid things when they get no response from the
DNS.

PS: I said in my earlier posting that anyone who wanted to see rate
limiting in BIND should feel free to contribute code. Since you seem
to think rate limiting DNS queries is a desirable thing to do, go
ahead. Implement it.

[Ladislav Vobr]

Post by Jim Reid
You obviously haven't understood what I posted. A firewall doesn't
only completely block unwanted traffic. Some firewalls *do* provide
rate limiting. As, of course, do routers.

hmm, perhaps you haven't understood what I posted as well, and I see
you reply is very general one, have you ever try to do such a thing? I
have not said that rate limiting is not in the firewalls or routers, I
was talking about dynamic rate limiting, not that for example I will
preconfigure in my router firewall that user from 1.2.3.4 can not exceed
256kbs. Can you imagine router config when you have around 4 class B for
your customers and each of them might flood you :-) ? Restricting total
traffic for them doesn't help at all, preconfiguring **each** of them
(let's say /32) in the router config, are you really suggesting this?

Most of the fw/routers don't support dynamic rate limiting, and many
developers know it and their applications implement it, since it is a
must today for big public environements.

Post by Jim Reid
Ladislav> Customers doing what they want, if bind can rate limit
Ladislav> them, they will ofcourse re-evaluate their behaviour,
Ladislav> because they will be forced to do it.
This is nonsense. First of all, the customers are probably not "doing
what they want". They're most likely doing what their ISP told them to
do a long time ago. Presumably neither the ISP or the customer at that
time had a clue about DNS operations and the pointless stupidity of

We never advise customers to do it, however imho they feel more secure
configuring their firewalls with dns udp traffic to their ISP only (us)
not to all internet dns servers. UDP statefull firewall will help, but
educating the customers, or make sure they upgrade and use it is
completely different and long term task.

Post by Jim Reid
How someone choses to configure rate limiting on their routers is up
to them. In all likelihood, the excessive traffic will be coming from
a small number of IP addresses, so it would be trivial to make the

hmm, what is small for you, do you know that today almost everybody has
at least isdn,dsl,cable ? Do you know that to fill the recursive-client
queue on bind is a piece of cake even for analog dial-up user? Do you
know, that bind doesn't even bother to log this or give you a hint why
and who doing this?

Post by Jim Reid
PS: I said in my earlier posting that anyone who wanted to see rate
limiting in BIND should feel free to contribute code. Since you seem
to think rate limiting DNS queries is a desirable thing to do, go
ahead. Implement it.

I am trying my best here to solve this, so far I don't have any solution
only some kind of workaround, which I can not really offer, since it has
lot of drawbacks, and myself I am not sure, if it's really good to do.

Ladislav

[Jim Reid]

Ladislav> well, and I see you reply is very general one, have you
Ladislav> ever try to do such a thing?

No, but I know people who *have* to do this and discussed approaches
to rate limiting with them.

Ladislav> I was talking about dynamic rate limiting

Nobody else was. The OP was talking about query rate limiting hooks
for BIND. There was no mention of dynamic rate limiting. Until you
raised this non sequitur.

Ladislav> Most of the fw/routers don't support dynamic rate
Ladislav> limiting, and many developers know it and their
Ladislav> applications implement it, since it is a must today for
Ladislav> big public environements.

This is utterly irrelevant to the original discussion. DNS service is
not an application in the same sense as an HTTP or SMTP server is an
application. The same goes for the respective protocols. And as I
already said, BIND does not have hooks for limiting inbound
queries. For DNS queries That job is best done by a router in front of
the name server.

Ladislav> hmm, what is small for you, do you know that today
Ladislav> almost everybody has at least isdn,dsl,cable ? Do you
Ladislav> know that to fill the recursive-client queue on bind is
Ladislav> a piece of cake even for analog dial-up user? Do you
Ladislav> know, that bind doesn't even bother to log this or give
Ladislav> you a hint why and who doing this?

<scarcasm mode>
No. What's dsl? Do you mean to say a name server needs to be
configured and tuned for the environment where it gets deployed?
Fancy that!
</scarcasm mode>

[Ladislav Vobr]

Post by Jim Reid
Nobody else was. The OP was talking about query rate limiting hooks
for BIND. There was no mention of dynamic rate limiting. Until you
raised this non sequitur.

hmm, jim imho he was talking about rate limiting of hosts which he
doesn't know, which is basicaly dynamic, in this case your solution to
use router where you have to hardcode the customer ip is not really
helpfull, believe it or not.

Post by Jim Reid
This is utterly irrelevant to the original discussion. DNS service is
not an application in the same sense as an HTTP or SMTP server is an
application. The same goes for the respective protocols. And as I
already said, BIND does not have hooks for limiting inbound
queries. For DNS queries That job is best done by a router in front of
the name server.

hmm, just think about it jim, it is about the same thing, not to let one
random guy to overload the service and make it non-responsive for
others, router doesn't help it (be it http or smtp or dns, udp or tcp.
Router does only overall traffic, which is useless since the service is
basically non-responding for the rest of users during the flood. Best
job for dns queries imho is dynamic rate limiting, which routers don't do.

Post by Jim Reid
Ladislav> hmm, what is small for you, do you know that today
Ladislav> almost everybody has at least isdn,dsl,cable ? Do you
Ladislav> know that to fill the recursive-client queue on bind is
Ladislav> a piece of cake even for analog dial-up user? Do you
Ladislav> know, that bind doesn't even bother to log this or give
Ladislav> you a hint why and who doing this?
<scarcasm mode>
No. What's dsl? Do you mean to say a name server needs to be
configured and tuned for the environment where it gets deployed?
Fancy that!
</scarcasm mode>

It's not really difficult to fill up recursive-client queue today, even
from the slowest line, believe it or not, and tuning for such cases is
curretly zero, there are two ways, either be quiet about it, or try to
do something about it.

Ladislav

[Kevin Darcy]

Post by Ladislav Vobr

Post by Jim Reid
Nobody else was. The OP was talking about query rate limiting hooks
for BIND. There was no mention of dynamic rate limiting. Until you
raised this non sequitur.

hmm, jim imho he was talking about rate limiting of hosts which he
doesn't know, which is basicaly dynamic, in this case your solution to
use router where you have to hardcode the customer ip is not really
helpfull, believe it or not.

Post by Jim Reid
This is utterly irrelevant to the original discussion. DNS service is
not an application in the same sense as an HTTP or SMTP server is an
application. The same goes for the respective protocols. And as I
already said, BIND does not have hooks for limiting inbound
queries. For DNS queries That job is best done by a router in front of
the name server.

hmm, just think about it jim, it is about the same thing, not to let one
random guy to overload the service and make it non-responsive for
others, router doesn't help it (be it http or smtp or dns, udp or tcp.
Router does only overall traffic, which is useless since the service is
basically non-responding for the rest of users during the flood. Best
job for dns queries imho is dynamic rate limiting, which routers don't do.

Post by Jim Reid
Ladislav> hmm, what is small for you, do you know that today
Ladislav> almost everybody has at least isdn,dsl,cable ? Do you
Ladislav> know that to fill the recursive-client queue on bind is
Ladislav> a piece of cake even for analog dial-up user? Do you
Ladislav> know, that bind doesn't even bother to log this or give
Ladislav> you a hint why and who doing this?
<scarcasm mode>
No. What's dsl? Do you mean to say a name server needs to be
configured and tuned for the environment where it gets deployed?
Fancy that!
</scarcasm mode>

It's not really difficult to fill up recursive-client queue today, even
from the slowest line, believe it or not, and tuning for such cases is
curretly zero, there are two ways, either be quiet about it, or try to
do something about it.

My 2 cents: the simple brute-force DNS (D)DoS attacks are probably
better foiled using some sort of dynamic rate-limiting in the router,
since then you drop the packets as soon and efficiently as possible.
However, there are more sophisticated DNS (D)DoS attacks possible,
including:
1. Querying a wide range of long-TTL names with the aim of filling up
the cache with junk, or
2. Querying names which are known to have unreachable nameservers,
broken delegations, or other forms of DNS nastiness, with the aim of
busying out the victim resolver with retries, error recovery, logging, etc.

These kinds of (D)DoS attacks might give more "bang for the buck" per
query and thus allow the attack to succeed even as it flies under the
radar of a router-based rate-limiting scheme. It might be impossible in
some scenarios (because the routers don't have access to the resolver's
state information) or at the very least cost-prohibitive, to put code in
the routers to foil such attacks and therefore might be better to put it
in the resolver code.

- Kevin

[Ladislav Vobr]

Post by Kevin Darcy
However, there are more sophisticated DNS (D)DoS attacks possible,
1. Querying a wide range of long-TTL names with the aim of filling up
the cache with junk, or
2. Querying names which are known to have unreachable nameservers,
broken delegations, or other forms of DNS nastiness, with the aim of
busying out the victim resolver with retries, error recovery, logging, etc.
These kinds of (D)DoS attacks might give more "bang for the buck" per
query and thus allow the attack to succeed even as it flies under the
radar of a router-based rate-limiting scheme. It might be impossible in
some scenarios (because the routers don't have access to the resolver's
state information) or at the very least cost-prohibitive, to put code in
the routers to foil such attacks and therefore might be better to put it
in the resolver code.

it is not so difficult to get bind amplify 1 udp packet hundred, two
hundred times, and it is done so quietly that nobody (administrators)
have a clue about it, no logs, no warnings. It is bind internal design.
I did simple test with some unreachable nameservers, for 1 request bind
sent 125 outgoing requests.

This kind of flooding is daily routine for many authoritative servers,
since their brothers :-) high rate recursive bind servers (who don't
cache timeouts, don't cache servfail, don't slow down with the time, and
don't provide all, what they cache,) send out 10,20, 100 ... times
amplified requests to the authoritative servers. Definitely there is
some misconfiguration in place but usually on the authoritative server
side (zone expired, misconfiguration, servfail, reachibility...), but
not on the recursive server side. What happens, providers blocks the
source of such floods, which are recursive bind nameservers, configured
as per the best recommendations, basically doing what bind developers
think is perfectly fine to do.

We have got blocked several times, because of excessive traffic from our
recursive bind servers to remote authoritative servers, what can we do
about it, when bind itself doesn't bother even to log unreachable
servers or the recursive queue details.

Does anybody know how to configure the firewall so it will not let the
random user to fill-up recursive-client queue or how to configure the
firewall so it will not let bind to flood random misconfigured
destination with it's full bandwidth and still provide the service to
the rest of users.

Misconfiguration, reachibility issue, zone expired these are daily
problems, they are not specially crafted hacker's packets, I don't know
about similar application, which will face daily problems with such a
incredible amount it's own and remote side resources and without any
warning (on the caching server side).

Ladislav

[phn]

Post by Ladislav Vobr

Post by Kevin Darcy
However, there are more sophisticated DNS (D)DoS attacks possible,
1. Querying a wide range of long-TTL names with the aim of filling up
the cache with junk, or
2. Querying names which are known to have unreachable nameservers,
broken delegations, or other forms of DNS nastiness, with the aim of
busying out the victim resolver with retries, error recovery, logging, etc.
These kinds of (D)DoS attacks might give more "bang for the buck" per
query and thus allow the attack to succeed even as it flies under the
radar of a router-based rate-limiting scheme. It might be impossible in
some scenarios (because the routers don't have access to the resolver's
state information) or at the very least cost-prohibitive, to put code in
the routers to foil such attacks and therefore might be better to put it
in the resolver code.

it is not so difficult to get bind amplify 1 udp packet hundred, two
hundred times, and it is done so quietly that nobody (administrators)
have a clue about it, no logs, no warnings. It is bind internal design.
I did simple test with some unreachable nameservers, for 1 request bind
sent 125 outgoing requests.
This kind of flooding is daily routine for many authoritative servers,
since their brothers :-) high rate recursive bind servers (who don't
cache timeouts, don't cache servfail, don't slow down with the time, and
don't provide all, what they cache,) send out 10,20, 100 ... times
amplified requests to the authoritative servers. Definitely there is
some misconfiguration in place but usually on the authoritative server
side (zone expired, misconfiguration, servfail, reachibility...), but
not on the recursive server side. What happens, providers blocks the
source of such floods, which are recursive bind nameservers, configured
as per the best recommendations, basically doing what bind developers
think is perfectly fine to do.
We have got blocked several times, because of excessive traffic from our
recursive bind servers to remote authoritative servers, what can we do
about it, when bind itself doesn't bother even to log unreachable
servers or the recursive queue details.
Does anybody know how to configure the firewall so it will not let the
random user to fill-up recursive-client queue or how to configure the
firewall so it will not let bind to flood random misconfigured
destination with it's full bandwidth and still provide the service to
the rest of users.

Use access-lists on recursive servers ( only allow your own hosts ),
have no-recursion on your auhorative servers. Is that what you mean ?

[Ladislav Vobr]

Post by phn

Post by Ladislav Vobr
Does anybody know how to configure the firewall so it will not let the
random user to fill-up recursive-client queue or how to configure the
firewall so it will not let bind to flood random misconfigured
destination with it's full bandwidth and still provide the service to
the rest of users.

Use access-lists on recursive servers ( only allow your own hosts ),
have no-recursion on your auhorative servers. Is that what you mean ?

well, I have all these things in place, when I said random user, I ment
random user from our valid block, which is of course valid to use
recursion. But still I really don't want him to send 1000req/sec of
nonsense.

second, i don't want bind to act like crazy and multiply perfectly valid
traffic (from the end user perspective) hundered or two hundered times
in some cases, and flood remote authoritative servers with it.

I admit it is not a piece of cake at all to implement it, but this
doesn't make the problem disappear. Maybe it will take some time to
realize that this problem is here already today.

Ladislav

[Danny Mayer]

Post by Ladislav Vobr

Post by Kevin Darcy
However, there are more sophisticated DNS (D)DoS attacks possible,
1. Querying a wide range of long-TTL names with the aim of filling up
the cache with junk, or
2. Querying names which are known to have unreachable nameservers,
broken delegations, or other forms of DNS nastiness, with the aim of
busying out the victim resolver with retries, error recovery, logging, etc.

Block recursion so you only respond to requests for which you are authorative.

Post by Ladislav Vobr

Post by Kevin Darcy
These kinds of (D)DoS attacks might give more "bang for the buck" per
query and thus allow the attack to succeed even as it flies under the
radar of a router-based rate-limiting scheme. It might be impossible in
some scenarios (because the routers don't have access to the resolver's
state information) or at the very least cost-prohibitive, to put code in
the routers to foil such attacks and therefore might be better to put it
in the resolver code.

it is not so difficult to get bind amplify 1 udp packet hundred, two
hundred times, and it is done so quietly that nobody (administrators)
have a clue about it, no logs, no warnings. It is bind internal design.
I did simple test with some unreachable nameservers, for 1 request bind
sent 125 outgoing requests.
This kind of flooding is daily routine for many authoritative servers,
since their brothers :-) high rate recursive bind servers (who don't
cache timeouts, don't cache servfail, don't slow down with the time, and
don't provide all, what they cache,) send out 10,20, 100 ... times
amplified requests to the authoritative servers. Definitely there is
some misconfiguration in place but usually on the authoritative server
side (zone expired, misconfiguration, servfail, reachibility...), but
not on the recursive server side. What happens, providers blocks the
source of such floods, which are recursive bind nameservers, configured
as per the best recommendations, basically doing what bind developers
think is perfectly fine to do.
We have got blocked several times, because of excessive traffic from our
recursive bind servers to remote authoritative servers, what can we do
about it, when bind itself doesn't bother even to log unreachable
servers or the recursive queue details.

You need to remember that the DNS protocol is stateless. A nameserver
considers each query on its own without regard to previous queries. The only
information it saves in cache are responses to its own queries to other
nameservers. To add rate limiting would require changes to store previous
query information. It also requires a massive increase in memory to remember
that information and well as longer lookup times while it checks for previous
queries from the same source. You really want to do this? It's faster to
just send back an answer to the query.

Danny

[Ladislav Vobr]

Post by Danny Mayer
You need to remember that the DNS protocol is stateless. A nameserver
considers each query on its own without regard to previous queries. The only
information it saves in cache are responses to its own queries to other
nameservers. To add rate limiting would require changes to store previous
query information. It also requires a massive increase in memory to remember
that information and well as longer lookup times while it checks for previous
queries from the same source. You really want to do this? It's faster to
just send back an answer to the query.

I believe there are dns products, which were designed with these points
in mind, bind is imho currently not. Don't you see that this kind of
traffic causing such problems is growing not lineary but almost
exponentially in today's public internet. Better logging will help,
since we can use external script to adjust the rules (be it firewall,
bogus, blackhole, iptables, ipfw) If bind itself doesn't give you a clue
what makes your recursive queue full, you really need sometimes a
crystal ball to find it out:-) If recursive bind doesn't tell you right
now I am flooding outside servers with hundered two hundered requests
for each *this particular client request* I received, again you are
blind, you can not blackhole it, you can not bogus it, you can not block
it....

Ladislav

[Jim Reid]

Nicolas> Is there a good way to monitore the life about Root
Nicolas> Server ? (ping ? dig ? other ?) in order to be warn
Nicolas> when there is a problem.

There is no point in doing this. So don't. Even if you did,
what are you going to do if you found a problem? [Which is
highly unlikely.] Who are you going to call? And would they
pay any attention to you?

Ladislav> it might be very simple way of saying you have a world
Ladislav> connectivity problems, I was doing it as well, maybe I
Ladislav> am still doing it, I have to check :-)

No you don't and this is not a good way of checking for "world
connectivity problems". Many of the root servers do anycasting. The
same IP address (well /24) is announced from many places on the
internet at once. So a poor RTT to one of these servers could be a
local routing or peering problem that has no bearing on a site's
"world connectivity". For example, there's an instance of the K root
server at Doha in Qatar. [See http://k.root-servers.org.] If your ISP
doesn't peer -- exchange routing info -- at that internet exchange,
queries from your net to k.root-servers.net could be going to London
or Frankfurt or....

Ladislav> when you run recursive servers, it might get very bussy
Ladislav> when the world connectivity is not there, it is very
Ladislav> difficult on current binds to figure out why it is
Ladislav> bussy, you can just guess, why you recursive queue is
Ladislav> full, it might be really different reasons, maybe single
Ladislav> nasty user, maybe lot of users with viruses, maybe
Ladislav> normal users but world connectivity problem....

So what? I fail to see how battering on the root servers can possibly
give someone any sort of insight into how busy or loaded their local
name servers are.

Ladislav> if you know immediatelly that you have this kind of
Ladislav> connectivity problems, you might possibly do some
Ladislav> action, like disable recursion, reload, and serve the
Ladislav> requests from the cache only, which is basically imho
Ladislav> better than having completely over-utilized server with
Ladislav> completely non-responsive service.

None of this is in any way relevant to monitoring the status of root
servers.

As I said before, DNS administrators should focus on making sure
*their* name servers were configured and operated correctly. This
would be much more helpful to everyone and a better use of resources
than monitoring the health of the root servers which are already
provisioned, configured and operated properly, as well as continually
monitored.

[Steve Sandau]

<content snipped... sorry>

Post by Jim Reid
As I said before, DNS administrators should focus on making sure
*their* name servers were configured and operated correctly. This
would be much more helpful to everyone and a better use of resources
than monitoring the health of the root servers which are already
provisioned, configured and operated properly, as well as continually
monitored.

Yes. The best thing any admin can do is make sure his/her own servers
are working well. In my experience, that usually takes care of a
majority of the problems. ;-)

Steve