====== Fluentd ======

It is a service to manage system logs. It takes logs from software (MySQL, Apache, the system...) and unify their format and send them to the required place.

Also called td-agent.

Before install: http://docs.fluentd.org/articles/before-install

By default, td-agent is configured to take logs from HTTP and route them to stdout (''/var/log/td-agent/td-agent.log''). You can post sample log records using the curl command: ''$ curl -X POST -d 'json={"json":"message"}' http://localhost:8888/debug.test''

<code>
$ /etc/init.d/td-agent start
$ /etc/init.d/td-agent stop
$ /etc/init.d/td-agent restart
$ /etc/init.d/td-agent status
</code>

Reload the configuration: ''sudo /etc/init.d/td-agent reload''

Configuration is in ''/etc/td-agent/td-agent.conf''

Directives into the config:
  * **source** directives determine the input sources.
  * **match** directives determine the output destinations.
  * **filter** directives determine the event processing pipelines.
  * **system** directives set system wide configuration.
  * **@include** directives include other files.

Each source directive must include a type parameter. The type parameter specifies which input plugin to use.

The source submits events into the Fluentd’s routing engine. An event consists of three entities: tag, time and record. The tag is a string separated by ‘.’s (e.g. myapp.access), and is used as the directions for Fluentd’s internal routing engine. 

The “match” directive looks for events with _match_ing tags and tells Fluentd what to do.


==== Plugins ====

Install plugins: ''$ /usr/sbin/td-agent-gem install fluent-plugin-xx''

  * For replication use the out_copy plugin.

=== Fluentd-ui ===

  * https://github.com/fluent/fluentd-ui
  * Install: $ /usr/sbin/td-agent-gem install -V fluentd-ui
  * Start: ''/opt/td-agent/embedded/bin/fluentd-ui start''

==== Python ====
Libraries: 
  * [[https://pypi.python.org/pypi/fluent-logger/|fluent-logger]]
  * [[https://pypi.python.org/pypi/pyfluent|pyfluent]]

=== Fluentd-logger ===
<code python>
from fluent import sender, event
sender.setup('debug.test', host='127.0.0.1', port=24224)
event.Event('json', {"json":"message from python"})
</code>

=== pyfluent ===
<code python>
import logging
from pyfluent.logging import SafeFluentHandler
handler = SafeFluentHandler('localhost', 24224, 'pyfluent')
handler.setLevel(logging.INFO)
logger = logging.getLogger()
logger.setLevel(logging.INFO)
logger.addHandler(handler)
logger.info('hello pyfluent!')
</code>

<code python>
import logging
import logging.config

logger = logging.getLogger(__name__)

logging.config.dictConfig({
    'version': 1,              
    'disable_existing_loggers': False,  # this fixes the problem

    'handlers': {
        'fluentd': {
            'level': 'DEBUG',
            'class': 'pyfluent.logging.SafeFluentHandler',
            'tag': 'test.prueba',
        },
    },
    'loggers': {
        '': {                  
            'handlers': ['fluentd'],        
            'level': 'INFO'
        }
    }
})

logger.info('It works!')
</code>