Syslog with BIND on CentOS

[John Thurston]

Many years ago, when we ran ISC BIND on Solaris, we created a logging
channel to send the logged-queries to the local syslogd. We then had our
local syslogd forward most of the traffic on to a central syslog server.

I just tried to re-implement something like that on CentOS, and thought
I had it working . . until it was exposed to full production traffic
load. The output to our central syslog server was truncated, and my
local system log was filled with messages saying jourald was activating
ratelimiting. !?

My subsequent read of the docs indicates that BIND on CentOS 7, while
being told it is sending to 'syslogd', is sending to 'journald' which is
handling all the messages and forwarding them on to 'syslogd'. I don't
want journald handling my thousands of messages per second from BIND. I
don't want that information in my journal logs. I just want it out in
the central syslog server.

Is there some direct way to get the logging channel of BIND pointed
directly into the local syslogd? (which would then apply its forwarding
rules to get traffic to the central syslog server)

I thought about trying to rip jourald out entirely, and quickly decided
that was a path to madness.

The only thing I can come up with is to activate dnstap, and have some
other process absorbing the data and spewing it directly to the central
syslogd.
--
--
Do things because you should, not just because you can.

John Thurston 907-465-8591
***@alaska.gov
Department of Administration
State of Alaska
_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information.


bind-users mailing list
bind-***@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

[Anand Buddhdev]

On 20/05/2021 23:34, John Thurston wrote:

Hi John,

Post by John Thurston
My subsequent read of the docs indicates that BIND on CentOS 7, while
being told it is sending to 'syslogd', is sending to 'journald' which is
handling all the messages and forwarding them on to 'syslogd'. I don't
want journald handling my thousands of messages per second from BIND. I
don't want that information in my journal logs. I just want it out in
the central syslog server.

On CentOS, journald listens on the syslog socket, and intercepts ALL log
messages, and logs them into files that are either in a memory-based
tmpfs (the default), or to disk (if you configure journald that way).
After intercepting the log message, and saving it to the journal,
journald then forwards the message to rsyslog, which listens on a
different socket.

Post by John Thurston
Is there some direct way to get the logging channel of BIND pointed
directly into the local syslogd? (which would then apply its forwarding
rules to get traffic to the central syslog server)

As far as I know, BIND just calls the syslog functions, and so the log
messages will go to whatever is listening on the default syslog socket
(journald on CentOS). I don't think there's any way to point BIND to
rsyslog's socket.

Post by John Thurston
I thought about trying to rip jourald out entirely, and quickly decided
that was a path to madness.

That is indeed the path to madness. On systemd-based servers, you can't
really do without journald.

Post by John Thurston
The only thing I can come up with is to activate dnstap, and have some
other process absorbing the data and spewing it directly to the central
syslogd.

You could also log directly to files (bypassing syslog), and then have
some process follow the files and send the logs to a remote server.

Regards,
Anand
_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information.


bind-users mailing list
bind-***@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

[Paul Kosinski via bind-users]

If you can have BIND log directly to a file, couldn't you use a FIFO (prwxrwxrwx) or Unix domain socket (srwxrwxrwx) and avoid the disk I/O by sending the log data directly to the forwarder? (E.g., Pulse Audio listens on a socket for audio data from an application, and sends it in real-time to the D/A hardware driver etc.)


On Fri, 21 May 2021 00:17:11 +0200

Post by Anand Buddhdev
Hi John,

Post by John Thurston
My subsequent read of the docs indicates that BIND on CentOS 7, while
being told it is sending to 'syslogd', is sending to 'journald' which is
handling all the messages and forwarding them on to 'syslogd'. I don't
want journald handling my thousands of messages per second from BIND. I
don't want that information in my journal logs. I just want it out in
the central syslog server.

On CentOS, journald listens on the syslog socket, and intercepts ALL log
messages, and logs them into files that are either in a memory-based
tmpfs (the default), or to disk (if you configure journald that way).
After intercepting the log message, and saving it to the journal,
journald then forwards the message to rsyslog, which listens on a
different socket.

Post by John Thurston
Is there some direct way to get the logging channel of BIND pointed
directly into the local syslogd? (which would then apply its forwarding
rules to get traffic to the central syslog server)

As far as I know, BIND just calls the syslog functions, and so the log
messages will go to whatever is listening on the default syslog socket
(journald on CentOS). I don't think there's any way to point BIND to
rsyslog's socket.

Post by John Thurston
I thought about trying to rip jourald out entirely, and quickly decided
that was a path to madness.

That is indeed the path to madness. On systemd-based servers, you can't
really do without journald.

Post by John Thurston
The only thing I can come up with is to activate dnstap, and have some
other process absorbing the data and spewing it directly to the central
syslogd.

You could also log directly to files (bypassing syslog), and then have
some process follow the files and send the logs to a remote server.
Regards,
Anand

_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information.


bind-users mailing list
bind-***@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

[John Thurston]

Post by Anand Buddhdev
You could also log directly to files (bypassing syslog), and then have
some process follow the files and send the logs to a remote server.

This seems rather inefficient, but there are established and flexible
tools to do just this.

Without changing the configuration of my named (which is currently
logging to a local file), I can make rsyslogd consider that file an
input source. Once in, the parsing and output modules can then work on it.

This relies on the input module "imfile", and the output module "omfwd"

https://rsyslog-doc.readthedocs.io/en/latest/configuration/modules/idx_input.html

imfile appears to follow log rotations cleanly. A limitation I see is
everything is assigned the same syslog facility.priority values.

It remains to be seen if this process can keep up with the query volume.

Warning: When started for the first time, imfile will read the existing
file and start forwarding. If the query log already contains 800MB of
lines, those will all be read in and passed through the parser and
output modules.

--
Do things because you should, not just because you can.

John Thurston 907-465-8591
***@alaska.gov
Department of Administration
State of Alaska
_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information.


bind-users mailing list
bind-***@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

[Petr Menšík]

Hello John,

I think it should be possible to use chroot and have there custom socket
mapped directly to rsyslog.

bind-chroot should be available in CentOS, try running
named-chroot.service instead of named.service.

I have not tried it on real installation, but I guess it should be
easiest way to use arbitrary socket different than common one.

Regards,
Petr

Post by John Thurston
Many years ago, when we ran ISC BIND on Solaris, we created a logging
channel to send the logged-queries to the local syslogd. We then had our
local syslogd forward most of the traffic on to a central syslog server.
I just tried to re-implement something like that on CentOS, and thought
I had it working . . until it was exposed to full production traffic
load. The output to our central syslog server was truncated, and my
local system log was filled with messages saying jourald was activating
ratelimiting. !?
My subsequent read of the docs indicates that BIND on CentOS 7, while
being told it is sending to 'syslogd', is sending to 'journald' which is
handling all the messages and forwarding them on to 'syslogd'. I don't
want journald handling my thousands of messages per second from BIND. I
don't want that information in my journal logs. I just want it out in
the central syslog server.
Is there some direct way to get the logging channel of BIND pointed
directly into the local syslogd? (which would then apply its forwarding
rules to get traffic to the central syslog server)
I thought about trying to rip jourald out entirely, and quickly decided
that was a path to madness.
The only thing I can come up with is to activate dnstap, and have some
other process absorbing the data and spewing it directly to the central
syslogd.