Network Device > Syslog-ng
> Logstash > Graylog2
Step 1
Cisco Device configuration for enabling Logging to a syslog server:
Global
configuration:
(config)#logging
trap debugging
logging
origin-id hostname
logging
facility syslog
logging host
{ip_address_of_syslog_server} transport udp port {1000} #####custom port for UDP
logging host
{ip_address_of_syslog_server} transport tcp port {1540} #####custom port for TCP
logging on
Juniper Device configuration for enabling Logging to a syslog server:
From Home screen Configuration>Report Settings>Syslog:
From Home screen Configuration>Report Settings>Syslog:
Step 2
Syslog Server configuration:
Install
syslog-ng on the linux
#apt-get
install syslog-ng
Step 3
Configure the syslog-ng.conf file for collecting and creating log files
# locate
syslog-ng.conf
#nano /etc/syslog-ng/syslog-ng.conf #######Default file location
Syslog-ng configuration File:
-------------------------------------------------------------------------------------------------------
Paste the below contents in syslog-ng.conf:
######
Source Network definition ################
source s_network_udp {
udp(ip(0.0.0.0) port(1000));
};
source s_network_tcp {
tcp(ip(0.0.0.0) port(1000)
max-connections(5000));
};
##########
Destination file Definition #########
##########
Create the path if not already there /var/log/HOSTS/ #########
destination d_clients {
file("/var/log/HOSTS/$HOST"); };
#########
Collecting logs from devices and writing to files ##########
#####
Source:s_network_udp and destination:d_clients defined in earlier steps
log {
source(s_network_udp); destination(d_clients); };
-----------------------------------------------------------------------------------------------------------
Save the
syslog-ng.conf and restart the service.
# service
syslog-ng start | Restart
_____________________________________________________
Above step will create log entry files for all the devices sending syslog messages on port 1000 to the directory /var/log/HOSTS/
Making directory:
Verify it with command:
/var/log/HOSTS/hostname.log
Above step will create log entry files for all the devices sending syslog messages on port 1000 to the directory /var/log/HOSTS/
Making directory:
# mkdir /var/log/HOSTS/
# chmod XXX /var/log/HOSTS/
Verify it with command:
# ls -ll /var/log/HOSTS
eg:- /var/log/HOSTS/x.x.x.x.log/var/log/HOSTS/hostname.log
Step 4
Download latest Logstash file logstash-x.x.x-monolithic.jar from the site http://logstash.net/
Logstash configuration:
Step 5
Create a new configuration file "mylogstash.conf". The configuration file name could be anything eg:-sharmith.conf :)Logstash configuration:
Logstash
configuration file Name: mylogstash.conf
# vi mylogstash.conf
-----------------------------------------------------------------------------------------------------------
###INPUT###
input {
file {
type => "Network"
path => [ "/var/log/HOSTS/**/*" ]
}
}
######## For
reflecting the real Hostnames in Graylog2 interface ##########
filter {
grok { match => [
"@source_path", "/var/log/HOSTS/%{HOSTNAME:host}" ] }
mutate { replace => [
"@source_host", "%{host}" ] }
}
output {
stdout {
}
elasticsearch {
embedded => true
}
gelf {
chunksize => 1420
# facility => "logstash-gelf" #########Default Setting
##########
facility => "%{@type}"
host => "127.0.0.1"
# level => "INFO" #########Default
Setting ##########
level => "%{level}"
port => 12201
sender => "%{@source_host}"
}
}
---------------------------------------------------------------------------------------------------------------
Step 6
Save the configuration and execute
# java -jar
logstash-1.1.0-monolithic.jar agent -f mylogstash.conf -- web --backend
elasticsearch:///?local&
To check the status of logstash service and start it automatically if it is not running.
1)
Create a Directory called "cron" in
the folder /opt/logstash
#mkdir cron
2)
Create a script file named
"logstash_check.sh" inside directory "/opt/logstash/cron"
#nano
logstash_check.sh
#### Paste the below portion
to the file. Save it and exit. ##########
#!/bin/bash
cd /opt/logstash
PATH=/usr/bin:/bin:/opt/logstash:/usr/sbin/:/sbin
clear
echo "Checking Logstash
service"
while true
do
sleep 5
# check the process every 5 seconds
echo "Sleep timer
passed"
if [ `pgrep -c -f
logstash-1.1.1-monolithic.jar` = 0 ]
echo "pgrep command
executed"
then
echo "problem!"
sendmail sharmithr@fss.co.in <
/opt/logstash/email_text.txt
java -jar logstash-1.1.1-monolithic.jar
agent -f mylogstash.conf -- web --backend elasticsearch:///?local&
echo "Mail sent to
administrator"
break
echo "After Break"
killall logstash_check.sh
else
echo "Logstash working"
killall logstash_check.sh
fi
done
3)
check the script by running it manually
#./logstash_check.sh
In
the paths directory mention the location of the commands by using the
"locate" command.
eg:- locate echo
locate sendmail
locate logstash-1.1.1-monolithic.jar
locate killall
4)
Issue the command on the command prompt before
creating the file. This will show the status output.
# pgrep
-c -f logstash-1.1.1-monolithic.jar
1
5)
Make the script run in the background
automatically with the help of cron process.
#crontab -e
#### Paste the below portion
to the file. Save it and exit. ##########
30 8 * * * /opt/logstash/cron/logstash_check.sh
***I have mentioned those “echo”
commands for verifying the working of the shell script. Upon your requirement “echo”
commands could be edited or removed.
Logstash Tips:
Step 7
Graylog2 Server, Graylog2 Web
Interface, Mongo DB and Elastic search configuration.
Refer the
Step by Step Guide mentioned in the below links.
#nano /opt/graylog2-web-interface/config/mongoid.yml
Make sure that there is no space before the "production:" Space will lead to errors.
Creating Virtual Host in Apache 2
Paste the below lines in /etc/apache2/apache2.conf at the bottom of the file.
###############
LoadModule passenger_module /usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.$
PassengerRoot /usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.12
PassengerRuby /usr/local/bin/ruby
################
Create a file called graylog2 in /etc/apache2/sites-available/ folder
Add the below lines as per your custom port in /etc/apache2/ports.conf
file:
-----------------------------------------------------------------------------
NameVirtualHost *:8080 ######## Custom Port for Graylog2
#######
Listen 8080
----------------------------------------------------------------------------
Restart Apache2 service
# service apache2 restart | start
For verifying whether the logs are coming to your server, run the below command: tcpdump package required.
Troubleshooting & fine tuning tips:
#tcpdump dst port {graylog2_Listen_port}
#tcpdump dst port 514
To make Graylog2 listen to custom port:
Required if you are running multiple services on the same server.
First verify the port status.
# ps aux | grep {port}
If there are no services running on the specific port, configure the port in /etc/graylog2.conf file as given below:
------------------------------------------------------------------------------------------------
# On which port (UDP) should we listen for Syslog messages? (Standard: 514)
syslog_listen_port =xxx
#syslog_listen_port = 5400 ###########example configuration
syslog_protocol = udp
------------------------------------------------------------------------------------------------
Restart Graylog2 Service
# service graylog2-server restart | start | stop
Any queries or difficulties in configuring this please feel free to send a note to sharmith@gmail.com. Enjoy :)
I got an error message while trying to restart Graylog2 server after a system reboot. It gave me the error message "FATAL: org.graylog2.Main - Could not start syslog server core thread. Do you have permissions to listen on port 514?"
I checked whether any other ports are listening to that port or not but found none and tried changing the port to another but with no luck.
Fixed the issue by killing the java process.
#ps aux | grep java
It listed a Java process as given below
root 1469 1.0 1.9 1049788 62860 ? Sl 10:10 0:19 /usr/bin/java -jar /opt/graylog2-server/graylog2-server.jar
Issued the command "kill 1469" to terminate the process and started the graylog2-server process and it worked fine.
Request everyone to put a comment about this blog and give your suggestions for improvement.
--------------------------------------------------------------------------------------------------------------
Disclaimer:
I got an error message while trying to restart Graylog2 server after a system reboot. It gave me the error message "FATAL: org.graylog2.Main - Could not start syslog server core thread. Do you have permissions to listen on port 514?"
I checked whether any other ports are listening to that port or not but found none and tried changing the port to another but with no luck.
Fixed the issue by killing the java process.
#ps aux | grep java
It listed a Java process as given below
root 1469 1.0 1.9 1049788 62860 ? Sl 10:10 0:19 /usr/bin/java -jar /opt/graylog2-server/graylog2-server.jar
Issued the command "kill 1469" to terminate the process and started the graylog2-server process and it worked fine.
Graylog2 Email notification
Configuration:
To get email notification for Alarms and Subscriptions you
need to enable email notification in Graylog2.
Main configuration files:
/opt/graylog2-web-interface/config/general.yml
/opt/graylog2-web-interface/config/email.yml
------------------------------------------------------------------------------------------------------------
Step 1)
Editing to be done in the files according to your SMTP
server and domain name:
#nano /opt/graylog2-web-interface/config/general.yml
######## Contents of the file ######################
general:
external_hostname: "your-graylog2.example.org" # Used for
example to generate permalinks. Don't add 'http://' or trailing slashes.
date_format:
"%d.%m.%Y - %H:%M:%S" #
http://ruby-doc.org/core/classes/Time.html#M000298 (strftime syntax)
allow_deleting:
false # Allowing deleting of messages negatively impacts performance
allow_version_check: true # Enables manual (/versioncheck/index) and
automatic (every 30min from overview page) version checking against
graylog2.org via HTTP.
#
custom_cookie_name: graylog2_staging1 # Set an own cookie name - Useful for
multiple deployments on same host like example.org/staging1/graylog2,
example.org/staging2/grayl$
# Settings for stream subscription emails.
subscriptions:
from:
graylog2subscriptions@example.org
subject:
"[graylog2] Subscription"
# Settings for stream alarm emails.
streamalarms:
from:
graylog2alarms@example.org
subject:
"[graylog2] Stream alarm!"
hoptoad:
enabled: false
ssl: false
api_key: 123
#nano /opt/graylog2-web-interface/config/email.yml
################Copy from here and paste it in the file
############
production:
via: smtp # via:
sendmail
host: 10.x.x.x
enable_starttls_auto: true
port: 25
####### If you don’t want authentication comment out the
next three lines ######
# auth: plain #
plain, login, cram_md5 - Comment out or remove to use no auth
# user: somebody
# password:
yourpass
domain:
example.org # the HELO domain provided by the client to the server
#########################################################
---------------------------------------------------------------------------------------------------------------
Step 2)
Create a script file for executing Stream Alarm check and
Stream Subscription check.
#nano run_rake_tasks_Alarms.sh
###### Copy the file and paste it into the file
#############
#!/bin/bash
cd /opt/graylog2-web-interface/
PATH=/usr/local/bin
rake RAILS_ENV=production streamalarms:send
########################################################
### Path of the rake command could be found by issuing the
command
#locate rake | more
#nano run_rake_tasks_ Subscriptions.sh
###### Copy the file and paste it into the file
#############
#!/bin/bash
cd /opt/graylog2-web-interface/
PATH=/usr/local/bin
rake RAILS_ENV=production subscriptions:send
########################################################
--------------------------------------------------------------------------------------------------------------
Step 3)
Create a cron file for executing your scripts based on your
requirement.
I am pasting the configuration for executing the scripts
every one minute.
#crontab –e
######## Copy and paste the below items to the cron file
bottom##########
* * * * *
/opt/graylog2-web-interface/cron/run_rake_tasks_Subscription.sh
* * * * *
/opt/graylog2-web-interface/cron/run_rake_tasks_Alarms.sh
--------------------------------------------------------------------------------------------------------------
Verify whether the below given screenshot is displayed in
the Home screen right corner showing the services as running.
Request everyone to put a comment about this blog and give your suggestions for improvement.
--------------------------------------------------------------------------------------------------------------
Disclaimer:
I cannot assume any liability for the content of
external pages. Solely the operators of those linked pages are
responsible for their content.
I make every reasonable effort to ensure that the
content of this Web site is kept up to date, and that it is accurate
and complete. Nevertheless, the possibility of errors cannot be entirely
ruled out. I do not give any warranty in respect of the timeliness,
accuracy or completeness of material published on this Web site, and
disclaim all liability for (material or non-material) loss or damage
incurred by third parties arising from the use of content obtained from
the Web site.
Registered trademarks and proprietary names, and
copyrighted text and images, are not generally indicated as such on my
Web pages. But the absence of such indications in no way implies the
these names, images or text belong to the public domain in the context
of trademark or copyright law.
All products and firm names used in this site are proprietary names of their corresponding owners. All rights are reserved which are not explicitly granted here.
