how two dns bind master sync?

[Zhengyu Pan]

Hi:
In my application scenario, I have two master. Each master connect several slave dns. When users update zone, i update these two master respectively in a for loop. However, when any master update fails, i will roll bock. you know, whenever any update, zone's serial will increase. this cause that the serial numbers of zone in two masters are inconsistent. How can i keep these two masters' zones consistent in real time? Is using rsync tool a good way? In the industry, is there a good way to synchronize two masters?


Best Regards
Zhengyu Pan

[Zhengyu Pan]

Because I need to a master as a cold standby. Another master need to have the same zone and configuration with the former. When a master is down, I need to switch to another master right away.


On 8/22/2018 16:39Reindl Harald<***@thelounge.net> wrote

Am 22.08.2018 um 09:15 schrieb Zhengyu Pan:
Hi:
In my application scenario, I have two master. Each master connect
several slave dns. When users update zone, i update these two master
respectively in a for loop. However, when any master update fails, i
will roll bock. you know, whenever any update, zone's serial will
increase. this cause that the serial numbers of zone in two masters are
inconsistent. How can i keep these two masters' zones consistent in real
time? Is using rsync tool a good way? In the industry, is there a good
way to synchronize two masters?
why two masters to begin with?

[Zhengyu Pan]

Thank you for your reply. How to switch slave to master in real time? Does bind have command?


On 8/22/2018 18:32Reindl Harald<***@thelounge.net> wrote


Am 22.08.2018 um 12:17 schrieb Zhengyu Pan:
Because I need to a master as a cold standby. Another master need to
have the same zone and configuration with the former. When a master
is down, I need to switch to another master right away.

you can make any slave at any point of time to a new master

typically you have *one* master and one slave which is confirured as
fallback-master on any other slaves - so in case you primary master
fails you change that one to a master with the data it already has just
make your changes on that slave which now became the master

when you restore the old master you just configure it as slave like
before and both just switching roles

there is no reason to maintain two masters

On 8/22/2018 16:39Reindl Harald<***@thelounge.net>
<mailto:***@thelounge.net> wrote


Am 22.08.2018 um 09:15 schrieb Zhengyu Pan:

Hi:
In my application scenario, I have two master. Each master connect
several slave dns. When users update zone, i update these two master
respectively in a for loop. However, when any master update fails, i
will roll bock. you know, whenever any update, zone's serial will
increase. this cause that the serial numbers of zone in two
masters are
inconsistent. How can i keep these two masters' zones consistent
in real
time? Is using rsync tool a good way? In the industry, is there
a good
way to synchronize two masters?

why two masters to begin with?

[Matus UHLAR - fantomas]

Post by Zhengyu Pan
Thank you for your reply. How to switch slave to master in real time?

If you store the data in a file, simply redefine the zone type, change
"type master;"
to
"type slave;"

If you don't keep slave zones in files, you are out of luck.

Post by Zhengyu Pan
Does bind have command?

no.

Post by Zhengyu Pan
Because I need to a master as a cold standby.

the question apparently was, why do you need it. This is not an answer to
that question.

Post by Zhengyu Pan
Another master need to
have the same zone and configuration with the former. When a master
is down, I need to switch to another master right away.

slaves can be slaves to multiple servers. Servers can have multiple roles.

ine server can fetch one zone from its master, but be a master to another
server for the same zone.

Post by Zhengyu Pan
In my application scenario, I have two master. Each master connect
several slave dns. When users update zone, i update these two master
respectively in a for loop. However, when any master update fails, i
will roll bock. you know, whenever any update, zone's serial will
increase. this cause that the serial numbers of zone in two
masters are
inconsistent. How can i keep these two masters' zones consistent in real
time? Is using rsync tool a good way? In the industry, is there a good
way to synchronize two masters?

The one example of this case it the active directory. It supports multiple
masters, BUT:
a DNS server should not be DNS slave to multiple AD servers. The AD servers
contain the data, but serial numbers are unrealiable in this case.

I'm afraid the same would apply for your application.
--
Matus UHLAR - fantomas, ***@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
WinError #98652: Operation completed successfully.
_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
bind-***@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

[潘政宇]

Post by Matus UHLAR - fantomas

Post by Zhengyu Pan
Thank you for your reply. How to switch slave to master in real time?

If you store the data in a file, simply redefine the zone type, change
"type master;" to
"type slave;¡±

However, I have hundreds of thousands of zone files in my master. If I modify these zone files sequentially and then reload bind, it will spend a lot of time. 

Post by Matus UHLAR - fantomas
If you don't keep slave zones in files, you are out of luck.

Post by Zhengyu Pan
Does bind have command?

no.

Post by Zhengyu Pan
Because I need to a master as a cold standby.

the question apparently was, why do you need it. This is not an answer to
that question.

Why I need the second master is that one master may send notify packet unsuccessfully with the increment of slave dns. I need that one part of slave servers connect the first master, the other part of slave servers connect the second master. So i need that these two master¡¯s zone are fully consistent.

Post by Matus UHLAR - fantomas

Post by Zhengyu Pan
Another master need to
have the same zone and configuration with the former. When a master
is down, I need to switch to another master right away.

slaves can be slaves to multiple servers. Servers can have multiple roles.
ine server can fetch one zone from its master, but be a master to another
server for the same zone.

How to make a zone is type master for slave dns and the same zone is slave for master in a server. How to configure it in named.conf?

Post by Matus UHLAR - fantomas

Post by Zhengyu Pan
In my application scenario, I have two master. Each master connect
several slave dns. When users update zone, i update these two master
respectively in a for loop. However, when any master update fails, i
will roll bock. you know, whenever any update, zone's serial will
increase. this cause that the serial numbers of zone in two
masters are
inconsistent. How can i keep these two masters' zones consistent in real
time? Is using rsync tool a good way? In the industry, is there a good
way to synchronize two masters?

The one example of this case it the active directory. It supports multiple
a DNS server should not be DNS slave to multiple AD servers. The AD servers
contain the data, but serial numbers are unrealiable in this case.
I'm afraid the same would apply for your application.
--
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
WinError #98652: Operation completed successfully.
_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list
bind-users mailing list
https://lists.isc.org/mailman/listinfo/bind-users

[Mark Andrews]

Post by 潘政宇

Post by Matus UHLAR - fantomas

Post by Zhengyu Pan
Thank you for your reply. How to switch slave to master in real time?

If you store the data in a file, simply redefine the zone type, change
"type master;" to
"type slave;”

However, I have hundreds of thousands of zone files in my master. If I modify these zone files sequentially and then reload bind, it will spend a lot of time. 

Automatic multi-master is really hard to do. Ask anyone that does
database replication.

Manual multi-master however is easier to do as you have a human in the
loop deciding which machine is master.

Switching a server from standby master to active master can be a simple as

cp /etc/named.active.conf /etc/named.conf
rndc reload

and

returning it to being a standby master

cp /etc/named.standby.conf /etc/named.conf
rndc reload

If you keep /etc/named.active.conf and /etc/named.standby.conf
preconfigured for the two mode of operation.

Since you are using dynamic update

active:
zone example.com {
type master;
masterfile-format raw;
update-policy {};
...
};

standby:
zone example.com {
type slave;
masterfile-format raw;
masters { other-master; all-the-machines-that-transfer-from-one-of-these-masters; };
update-forward { … };
...
};

then you use rsync to keep /etc/named.active.conf and /etc/named.standby.conf up to date.

You use all the slaves as masters to the standby master so that you have a very
low probability of a lost update due to the active master dying before the update
makes it onto one of the slaves. Named will almost certainly be up to date on
the standby master by the time you go to promote it to active master. This can
be checked by checking all the serial numbers for the zones.

You also need to be able to detect whether that other master is in active or
standby mode when you the server reboots. If you can’t determine this put the
server into standby mode. I would use

zone standby-mode {
type master;
file “active.db”;
};

zone standby-mode {
type master;
file “standby.db”;
};

as a way to do this and cross checking.

Post by 潘政宇

Post by Matus UHLAR - fantomas
If you don't keep slave zones in files, you are out of luck.

Post by Zhengyu Pan
Does bind have command?

no.

Post by Zhengyu Pan
Because I need to a master as a cold standby.

the question apparently was, why do you need it. This is not an answer to
that question.

Why I need the second master is that one master may send notify packet unsuccessfully with the increment of slave dns. I need that one part of slave servers connect the first master, the other part of slave servers connect the second master. So i need that these two master’s zone are fully consistent.

Post by Matus UHLAR - fantomas

Post by Zhengyu Pan
Another master need to
have the same zone and configuration with the former. When a master
is down, I need to switch to another master right away.

slaves can be slaves to multiple servers. Servers can have multiple roles.
ine server can fetch one zone from its master, but be a master to another
server for the same zone.

How to make a zone is type master for slave dns and the same zone is slave for master in a server. How to configure it in named.conf?

Post by Matus UHLAR - fantomas

Post by Zhengyu Pan
In my application scenario, I have two master. Each master connect
several slave dns. When users update zone, i update these two master
respectively in a for loop. However, when any master update fails, i
will roll bock. you know, whenever any update, zone's serial will
increase. this cause that the serial numbers of zone in two
masters are
inconsistent. How can i keep these two masters' zones consistent in real
time? Is using rsync tool a good way? In the industry, is there a good
way to synchronize two masters?

The one example of this case it the active directory. It supports multiple
a DNS server should not be DNS slave to multiple AD servers. The AD servers
contain the data, but serial numbers are unrealiable in this case.
I'm afraid the same would apply for your application.
--
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
WinError #98652: Operation completed successfully.
_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list
bind-users mailing list
https://lists.isc.org/mailman/listinfo/bind-users

_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list
bind-users mailing list
https://lists.isc.org/mailman/listinfo/bind-users

--
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: [phone] INTERNET: ***@isc.org

_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
bind-***@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind

[Grant Taylor via bind-users]

     In my application scenario, I have two master. Each master connect
several slave dns. When users update zone, i update these two master
respectively in a for loop. However, when any master update fails, i
will roll bock. you know, whenever any update, zone's serial will
increase. this cause that the serial numbers of zone in two masters are
inconsistent. How can i keep these two masters' zones consistent in real
time? Is using rsync tool a good way?  In the industry, is there a good
way to synchronize two masters?

This may be an unpopular opinion, especially on the BIND-Users mailing
list (sometimes BIND is not the best answer).

It sounds like you might want something like multi-master DNS servers
that Active Directory (with AD integrated zones) provides.

You can "Enable BIND secondaries" to allow (any) slave server to do a
standard zone transfer.

You could then make your change to one master DNS server and AD will
ensure that the other gets it too. Either way, without reconfiguring
anything.

I would love to see this type of feature in BIND. But I've not seen
anything provide it yet.

[Bob McDonald]

Post by Grant Taylor via bind-users
This may be an unpopular opinion, especially on the BIND-Users mailing
list (sometimes BIND is not the best answer).
It sounds like you might want something like multi-master DNS servers
that Active Directory (with AD integrated zones) provides.

Here's the Microsoft AD DNS explanation:
https://docs.microsoft.com/en-us/windows-server/identity/ad-ds/plan/active-directory-integrated-dns-zones

This may be the time to start some dialogue around the way Bind processes
updates. While AD integrated DNS does process updates for multiple masters,
it does it outside the Bind-centric communications path. (I believe it uses
AD to forward updates from one master to the others). Bind needs some sort
of multi-master framework but there are a few issues if things stay the way
they are. There are obvious issues with serial number accounting and slave
notification. There are also issues with update processing (and
forwarding). Right now the only server that can accept updates is the
master. Forwarded updates are stamped as coming from the forwarding node.
That makes tracking updates almost impossible. (And that seems to be the
case for both signed and un-signed updates) I may be not seeing something
but from my point of view, that, above all else, must change if a
meaningful multi-master framework is to emerge.

Regards,

Bob

[Barry S. Finkel]

Post by Bob McDonald

Post by Grant Taylor via bind-users
This may be an unpopular opinion, especially on the BIND-Users mailing
list (sometimes BIND is not the best answer).
It sounds like you might want something like multi-master DNS servers
that Active Directory (with AD integrated zones) provides.

https://docs.microsoft.com/en-us/windows-server/identity/ad-ds/plan/active-directory-integrated-dns-zones
This may be the time to start some dialogue around the way Bind processes
updates. While AD integrated DNS does process updates for multiple masters,
it does it outside the Bind-centric communications path. (I believe it uses
AD to forward updates from one master to the others). Bind needs some sort
of multi-master framework but there are a few issues if things stay the way
they are. There are obvious issues with serial number accounting and slave
notification. There are also issues with update processing (and
forwarding). Right now the only server that can accept updates is the
master. Forwarded updates are stamped as coming from the forwarding node.
That makes tracking updates almost impossible. (And that seems to be the
case for both signed and un-signed updates) I may be not seeing something
but from my point of view, that, above all else, must change if a
meaningful multi-master framework is to emerge.
Regards,
Bob

As I wrote many years ago when I had MS AD DNS Servers as slaves to my
BIND servers - See KB28286. With multi-master servers, it is not clear
what an updated zone serial number should be. Take this example:

A zone ad.example.com is mastered on two AD DNS Servers. Each one has
the same contents and serial number, say 100. Then, at the same time
one update comes in to each server. Each server performs the update
and updates the serial number to 101. But each server now has a
different version of the 101-serial zone. Somehow, under the covers,
AD synchronizes the zones so that they have the same content. What
should the serial number be for this combined zone? It can't be 102,
because during the synchronization process another update may have come
into one of those servers, causing the serial number there to have been
increased to 102. I have no idea what the new serial number should be.

That is why I chose ONE of the several MS AD DNS Servers as the "master"
to my BIND slave servers. And NO MS machine used the MS AD DNS Servers
as its DNS Servers; all were configured to use my BIND servers as their
DNS servers. That way I did not care what the serial number was on the
other AD DNS servers that were not the master for my BIND slaves.

And, as another related issue, there were times when the serial number
of an AD zone decreased during times when that Domain Controller was
being patched.

--Barry Finkel


_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
bind-***@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

[Grant Taylor via bind-users]

Somehow, under the covers, AD synchronizes the zones so that they have
the same content.

It's my understanding that MS-DNS servers hosting AD Integrated zones
are actually functioning as application layer gateways between DNS and
data that's stored in LDAP.

So the case of synchronizing records with different FQDNs is actually
trivial in that different records are being updated in the back end LDAP
and the ALG is simply reading the data and replying to clients.

I don't know how to account for the serial number. I think I've seen
something like an encoded form of the date / time be used. ¯\_(ツ)_/¯

[Grant Taylor via bind-users]

Post by Grant Taylor via bind-users
It's my understanding that MS-DNS servers hosting AD Integrated zones
are actually functioning as application layer gateways between DNS and
data that's stored in LDAP.

My AD Guy confirms that the DNS data for Active Directory Integrated
Zones is indeed stored in LDAP and that MS-DNS is acting as an
application layer gateway between DNS and LDAP. As such, the
multi-master aspect issue is pushed to AD's LDAP implementation.

Post by Grant Taylor via bind-users
So the case of synchronizing records with different FQDNs is actually
trivial in that different records are being updated in the back end LDAP
and the ALG is simply reading the data and replying to clients.

He confirmed that LDAP does support writes to different data on
different servers without a problem.

He even indicated that updates for the same FQDN may not be a problem,
depending on the operation being done. I.e. multiple inserts for A
records will simply merge in LDAP data. The thing he wasn't quite sure
of was what would happen if one server deletes an A record and another
server enters an A record. He thinks that LDAP will delete the first
record which is different and insert the other record.

He also mentioned that it is unlikely that the same FQDN would be
modified on two different servers at the same time. As such, LDAP would
likely see different FQDNs and simply merge them as part of the raw data.

This is where I wash my hands and decide that I want to NOT get any
deeper into AD.

[Darcy, Kevin]

As someone who has had to deal with the interaction between BIND and
AD-integrated DNS for most of my DNS career, I think it's important, from a
BIND perspective, to understand how a given AD-integrated DNS zone is used.
If clients are registering themselves in the AD zone, then there is going
to be a lot of "churn" in the zone, and all of these problems that worry
BIND people -- like "floating" serial numbers, SOA.MNAMEs that flip from
one DC to another, potential record-level inconsistencies between
"multi-masters", and (apparently) no good "resolution protocol" to
arbitrate between them, most likely come to the forefront.

For us, we made the decision early on that we were *not* going to register
clients in our AD zones. So, what's in those zones mostly consists of SRV
records for the service-location function (sometimes called "DC Locator" in
Microsoft-ese). About the only A records are for the domain controllers
themselves. None of this data changes very frequently, so the SOA-related
issues and the potential multi-master consistency issues really haven't
bitten us. We replicate the AD zone data to our BIND-based infrastructure,
and this allows us to reap the benefits of things like Anycast-based
resolution, decent query logging and so forth. We've even managed to tweak
the NOTIFYs to some degree, in order for the replication to occur in a
timely fashion.

YMMV if you register your clients in AD, but for us, it's been mostly a
peaceful co-existence. About the biggest problem we have is a lack of
coordination when domain controllers are moved around, firewall rules are
botched, etc. But that's more of a big-company, chaos/silo-ing issue than a
technical challenge _per_se_. We compensate by monitoring the logs for
zone-transfer failure messages.


- Kevin

On Thu, Aug 23, 2018 at 6:50 PM, Grant Taylor via bind-users <

It's my understanding that MS-DNS servers hosting AD Integrated zones are
actually functioning as application layer gateways between DNS and data
that's stored in LDAP.

My AD Guy confirms that the DNS data for Active Directory Integrated Zones
is indeed stored in LDAP and that MS-DNS is acting as an application layer
gateway between DNS and LDAP. As such, the multi-master aspect issue is
pushed to AD's LDAP implementation.
So the case of synchronizing records with different FQDNs is actually

trivial in that different records are being updated in the back end LDAP
and the ALG is simply reading the data and replying to clients.

He confirmed that LDAP does support writes to different data on different
servers without a problem.
He even indicated that updates for the same FQDN may not be a problem,
depending on the operation being done. I.e. multiple inserts for A records
will simply merge in LDAP data. The thing he wasn't quite sure of was what
would happen if one server deletes an A record and another server enters an
A record. He thinks that LDAP will delete the first record which is
different and insert the other record.
He also mentioned that it is unlikely that the same FQDN would be modified
on two different servers at the same time. As such, LDAP would likely see
different FQDNs and simply merge them as part of the raw data.
This is where I wash my hands and decide that I want to NOT get any deeper
into AD.
--
Grant. . . .
unix || die
_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to
unsubscribe from this list
bind-users mailing list
https://lists.isc.org/mailman/listinfo/bind-users

[Dave Warren]

Post by Grant Taylor via bind-users

Somehow, under the covers, AD synchronizes the zones so that they have
the same content.

It's my understanding that MS-DNS servers hosting AD Integrated zones
are actually functioning as application layer gateways between DNS and
data that's stored in LDAP.
So the case of synchronizing records with different FQDNs is actually
trivial in that different records are being updated in the back end LDAP
and the ALG is simply reading the data and replying to clients.
I don't know how to account for the serial number.  I think I've seen
something like an encoded form of the date / time be used.  ¯\_(ツ)_/¯

tl;dr: The Active Directory logic for the serial number is simple: Each
update to the zone increments the serial as you expect locally. When DCs
sync they use the highest serial number of either, +1.

Imagine you have 3 (or more) servers, updates happening everywhere. The
is an unknown period of latency between updates, not all DCs receive
updates at the same time, and not all DCs receive updates in the same
order or use the same schedule.

Start off with serial 100.
Server1 updates, increments to serial 101.
Server1 updates, increments to serial 102.
Server1 updates, increments to serial 103.
Server2 updates, increments to serial 102.
Server1 updates, increments to serial 104.
Server3 updates, increments to serial 102.

At this point all three have different views of the zone, with serials
104, 102, 102 respectively.

Server2 and Server3 now sync, highest serial is 102 so both are set to
103. Server1 and Server2 now sync, highest serial is 104 but there are
other changes, so both increment to 105. Eventually Server1 and Server3
sync, highest serial is 105, but no other records have changed so both
agree to 105 and now everything is in sync once again.

Neither Active Directory nor Microsoft DNS care what is in the SOA
record in terms of the refresh, retry and expire intervals, DNS records
will instead synchronize around Active Directory along with every other
type of change.

If you set BIND slaves to master off of just server1 there are no
problems at all as the serial increases in a predictable and normal way.
If your BIND slaves off of multiple AD DCs then it will 1) Periodically
see and complain about (log) older serial numbers, and 2) Perodically
flip between the zone as reported by the different servers.

However, a lot of this happens if the DCs are sending notifies to BIND
and BIND is then changing it's mind about which master to use. If you
only have one DC send notifies then (I believe) BIND will tend to just
pull updates from that one DC and everybody is happy. If that DC goes
down then BIND will eventually hit the refresh interval and (if
configured) try other masters.

Happy weekend all!
_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
bind-***@lists.isc.org
https://lists.isc.org/mailman