It is a service to manage system logs. It takes logs from software (MySQL, Apache, the system…) and unify their format and send them to the required place.

Also called td-agent.

Before install: http://docs.fluentd.org/articles/before-install

By default, td-agent is configured to take logs from HTTP and route them to stdout (/var/log/td-agent/td-agent.log). You can post sample log records using the curl command: $ curl -X POST -d 'json={“json”:“message”}' http://localhost:8888/debug.test

$ /etc/init.d/td-agent start
$ /etc/init.d/td-agent stop
$ /etc/init.d/td-agent restart
$ /etc/init.d/td-agent status

Reload the configuration: sudo /etc/init.d/td-agent reload

Configuration is in /etc/td-agent/td-agent.conf

Directives into the config:

• source directives determine the input sources.
• match directives determine the output destinations.
• filter directives determine the event processing pipelines.
• system directives set system wide configuration.
• @include directives include other files.

Each source directive must include a type parameter. The type parameter specifies which input plugin to use.

The source submits events into the Fluentd’s routing engine. An event consists of three entities: tag, time and record. The tag is a string separated by ‘.’s (e.g. myapp.access), and is used as the directions for Fluentd’s internal routing engine.

The “match” directive looks for events with _match_ing tags and tells Fluentd what to do.

Install plugins: $ /usr/sbin/td-agent-gem install fluent-plugin-xx

• For replication use the out_copy plugin.

• https://github.com/fluent/fluentd-ui
• Install: $ /usr/sbin/td-agent-gem install -V fluentd-ui
• Start: /opt/td-agent/embedded/bin/fluentd-ui start

Libraries:

• fluent-logger
• pyfluent

from fluent import sender, event
sender.setup('debug.test', host='127.0.0.1', port=24224)
event.Event('json', {"json":"message from python"})

import logging
from pyfluent.logging import SafeFluentHandler
handler = SafeFluentHandler('localhost', 24224, 'pyfluent')
handler.setLevel(logging.INFO)
logger = logging.getLogger()
logger.setLevel(logging.INFO)
logger.addHandler(handler)
logger.info('hello pyfluent!')

import logging
import logging.config
 
logger = logging.getLogger(__name__)
 
logging.config.dictConfig({
    'version': 1,              
    'disable_existing_loggers': False,  # this fixes the problem
 
    'handlers': {
        'fluentd': {
            'level': 'DEBUG',
            'class': 'pyfluent.logging.SafeFluentHandler',
            'tag': 'test.prueba',
        },
    },
    'loggers': {
        '': {                  
            'handlers': ['fluentd'],        
            'level': 'INFO'
        }
    }
})
 
logger.info('It works!')