Question: A file seems to be disappearing and no direct reason for its disappearance can be found. How can we monitor the file access, especially the deletion of a particular file in CentOS/RHEL system?
A file can be monitored through its lifecycle by using the Linux auditd service.
The Steps
1. To install it on CentOS or RHEL, run:
# yum install audit
2. To enable the service at boot time, run:
# chkconfig auditd on # For CentOS/RHEL 6 # systemctl enable auditd # For CentOS/RHEL 7
or start it temporarily, until the next reboot:
# service auditd start # For CentOS/RHEL 6 # systemctl start auditd # For CentOS/RHEL 7
3. To watch a specific file to see if something touches the file, run:
# auditctl -w [filename] -p rwxa -k [filter key]
where:
-w [filename] : watch the specific file.
-p war : permissions filter for write, append, read.
-k [filter key] : a filter key to uniquely identify the audit logs.
For Example:
# auditctl -w /var/opt/sun/comms/messaging64/log/mail.log_yesterday -p rwxa -k mail.log_yesterday
4. To list the current set of rules, run:
# auditctl -l
For Example:
# auditctl -l LIST_RULES: exit,always watch=/var/opt/sun/comms/messaging64/log/mail.log_yesterday perm=rwxa key=mail.log_yesterday
Verify
To find out who changed or accessed the watched file:
# ausearch -i -k [filter key]
For Example:
# ausearch -i -k mail.log_yesterday ... ---- type=PATH msg=audit(12/12/2013 00:30:00.407:6209) : item=0 name=/opt/sun/comms/messaging64/data/log/mail.log_yesterday inode=425831 dev=fd:00 mode=file,600 ouid=mailsrv ogid=mail rdev=00:00 type=CWD msg=audit(12/12/2013 00:30:00.407:6209) : cwd=/var/opt/sun/comms/messaging64/log type=SYSCALL msg=audit(12/12/2013 00:30:00.407:6209) : arch=x86_64 syscall=open success=yes exit=5 a0=7fff62973b1d a1=0 a2=7fff62973870 a3=2 items=1 ppid=16442 pid=16450 auid=root uid=mailsrv gid=mail euid=mailsrv suid=mailsrv fsuid=mailsrv egid=mail sgid=mail fsgid=mail tty=(none) ses=570 comm=cat exe=/bin/cat key=mail.log_yesterday ---- type=CONFIG_CHANGE msg=audit(12/12/2013 00:30:00.407:6210) : auid=root ses=570 op="updated rules" path=/var/opt/sun/comms/messaging64/log/mail.log_yesterday key=mail.log_yesterday list=exit res=1 ---- type=PATH msg=audit(12/12/2013 00:30:00.407:6211) : item=1 name=/opt/sun/comms/messaging64/data/log/mail.log_yesterday inode=425831 dev=fd:00 mode=file,600 ouid=mailsrv ogid=mail rdev=00:00 type=PATH msg=audit(12/12/2013 00:30:00.407:6211) : item=0 name=/opt/sun/comms/messaging64/data/log/ inode=423325 dev=fd:00 mode=dir,700 ouid=mailsrv ogid=mail rdev=00:00 type=CWD msg=audit(12/12/2013 00:30:00.407:6211) : cwd=/var/opt/sun/comms/messaging64/log type=SYSCALL msg=audit(12/12/2013 00:30:00.407:6211) : arch=x86_64 syscall=unlink success=yes exit=0 a0=7ffff94beb1f a1=7ffff94beb1f a2=2 a3=2 items=2 ppid=16442 pid=16452 auid=root uid=mailsrv gid=mail euid=mailsrv suid=mailsrv fsuid=mailsrv egid=mail sgid=mail fsgid=mail tty=(none) ses=570 comm=rm exe=/bin/rm key=mail.log_yesterday ---- type=CONFIG_CHANGE msg=audit(12/12/2013 00:30:00.407:6212) : auid=root ses=570 op="updated rules" path=/var/opt/sun/comms/messaging64/log/mail.log_yesterday key=mail.log_yesterday list=exit res=1 ---- type=PATH msg=audit(12/12/2013 00:30:00.407:6213) : item=3 name=/opt/sun/comms/messaging64/data/log/mail.log_yesterday inode=425765 dev=fd:00 mode=file,600 ouid=mailsrv ogid=mail rdev=00:00 type=PATH msg=audit(12/12/2013 00:30:00.407:6213) : item=2 name=/opt/sun/comms/messaging64/data/log/mail.log_current inode=425765 dev=fd:00 mode=file,600 ouid=mailsrv ogid=mail rdev=00:00 type=PATH msg=audit(12/12/2013 00:30:00.407:6213) : item=1 name=/opt/sun/comms/messaging64/data/log/ inode=423325 dev=fd:00 mode=dir,700 ouid=mailsrv ogid=mail rdev=00:00 type=PATH msg=audit(12/12/2013 00:30:00.407:6213) : item=0 name=/opt/sun/comms/messaging64/data/log/ inode=423325 dev=fd:00 mode=dir,700 ouid=mailsrv ogid=mail rdev=00:00 type=CWD msg=audit(12/12/2013 00:30:00.407:6213) : cwd=/var/opt/sun/comms/messaging64/log type=SYSCALL msg=audit(12/12/2013 00:30:00.407:6213) : arch=x86_64 syscall=rename success=yes exit=0 a0=7fff81d5caea a1=7fff81d5cb1f a2=0 a3=0 items=4 ppid=16442 pid=16453 auid=root uid=mailsrv gid=mail euid=mailsrv suid=mailsrv fsuid=mailsrv egid=mail sgid=mail fsgid=mail tty=(none) ses=570 comm=mv exe=/bin/mv key=mail.log_yesterday ...
Here we see the rotation of mail.log files occurring, which is being done by the ‘return_job‘ which is happening by default at 00:30, and which is ran by user “mailsrv“. You can see the mail.log_yesterday file being cat (into the mail.log), being removed, and the mail.log_current getting renamed to the mail.log_yesterday.