# ECS Logging with Winston

Serverless

Stack

This Node.js package provides a formatter for the [winston](https://github.com/winstonjs/winston#readme) logger, compatible with [Elastic Common Schema (ECS) logging](/content/docs/reference/ecs/logging/intro/index.html). In combination with the [Filebeat](/content/beats/filebeat/index.html) shipper, you can [monitor all your logs](/content/log-monitoring/index.html) in one place in the Elastic Stack. `winston` 3.x versions >=3.3.3 are supported.

## Setup

### Step 1: Install

```cmd
$ npm install @elastic/ecs-winston-format
```

### Step 2: Configure

```js
const winston = require('winston');
const { ecsFormat } = require('@elastic/ecs-winston-format');

const logger = winston.createLogger({
  format: ecsFormat(/* options */),
  transports: [
    new winston.transports.Console()
  ]
});

logger.info('hi');
logger.error('oops there is a problem', { err: new Error('boom') });
```

1. Pass the ECS formatter to winston here.

### Step 3: Configure Filebeat

The best way to collect the logs once they are ECS-formatted is with [Filebeat](/content/docs/reference/beats/filebeat/index.html):

#### For Filebeat 7.16+

```yaml
filebeat.inputs:
- type: filestream
  paths: /path/to/logs.json
  parsers:
    - ndjson:
      overwrite_keys: true
      add_error_key: true
      expand_keys: true

processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~
```

#### For Filebeat < 7.16

```yaml
filebeat.inputs:
- type: log
  paths: /path/to/logs.json
  json.keys_under_root: true
  json.overwrite_keys: true
  json.add_error_key: true
  json.expand_keys: true

processors:
- add_host_metadata: ~
- add_cloud_metadata: ~
- add_docker_metadata: ~
- add_kubernetes_metadata: ~
```

### Kubernetes

1. Make sure your application logs to stdout/stderr.
2. Follow the [Run Filebeat on Kubernetes](/content/docs/reference/beats/filebeat/running-on-kubernetes/index.html) guide.
3. Enable [hints-based autodiscover](/content/docs/reference/beats/filebeat/configuration-autodiscover-hints/index.html).
4. Add these annotations to your pods that log using ECS loggers.

```yaml
annotations:
  co.elastic.logs/json.overwrite_keys: true
  co.elastic.logs/json.add_error_key: true
  co.elastic.logs/json.expand_keys: true
```

### Docker

1. Make sure your application logs to stdout/stderr.
2. Follow the [Run Filebeat on Docker](/content/docs/reference/beats/filebeat/running-on-docker/index.html) guide.
3. Enable [hints-based autodiscover](/content/docs/reference/beats/filebeat/configuration-autodiscover-hints/index.html).
4. Add these labels to your containers that log using ECS loggers.

```yaml
labels:
  co.elastic.logs/json.overwrite_keys: true
  co.elastic.logs/json.add_error_key: true
  co.elastic.logs/json.expand_keys: true
```

## Usage

```js
const winston = require('winston');
const { ecsFormat } = require('@elastic/ecs-winston-format');

const logger = winston.createLogger({
  level: 'info',
  format: ecsFormat(/* options */),
  transports: [
    new winston.transports.Console()
  ]
});

logger.info('hi');
logger.error('oops there is a problem', { foo: 'bar' });
```

## Error logging

By default, the formatter will convert an `err` meta field that is an Error instance to [ECS Error fields](/content/docs/reference/ecs/ecs-error/index.html). For example:

```js
const winston = require('winston');
const { ecsFormat } = require('@elastic/ecs-winston-format');
const logger = winston.createLogger({
  format: ecsFormat(),
  transports: [
    new winston.transports.Console()
  ]
});

const myErr = new Error('boom');
logger.info('oops', { err: myErr });
```

will yield (pretty-printed for readability):

```cmd
% node examples/error.js | jq .
{
  "@timestamp": "2021-01-26T17:25:07.983Z",
  "log.level": "info",
  "message": "oops",
  "error": {
    "type": "Error",
    "message": "boom",
    "stack_trace": "Error: boom\n    at Object.<anonymous> (..."
  },
  "ecs.version": "8.10.0"
}
```

## HTTP Request and Response Logging

With the `convertReqRes: true` option, the formatter will automatically convert Node.js core request and response objects when passed as the `req` and `res` meta fields, respectively.

```js
const http = require('http');
const winston = require('winston');
const { ecsFormat } = require('@elastic/ecs-winston-format');

const logger = winston.createLogger({
  level: 'info',
  format: ecsFormat({ convertReqRes: true }),
  transports: [
    new winston.transports.Console()
  ]
});

const server = http.createServer(handler);
server.listen(3000, () => {
  logger.info('listening at http://localhost:3000')
});

function handler (req, res) {
  res.setHeader('Foo', 'Bar');
  res.end('ok');
  logger.info('handled request', { req, res });
}
```

## Log Correlation with APM

This ECS log formatter integrates with [Elastic APM](/content/apm/index.html). If your Node app is using the [Node.js Elastic APM Agent](/content/docs/reference/apm/agents/nodejs/index.html), then a number of fields are added to log records to correlate between APM services or traces and logging data:

- Log statements (e.g. `logger.info(...)`) called when there is a current tracing span will include [tracing fields](/content/docs/reference/ecs/ecs-tracing/index.html)—`trace.id`, `transaction.id`, `span.id`.
- A number of service identifier fields determined by or configured on the APM agent allow cross-linking between services and logs in Kibana—`service.name`, `service.version`, `service.environment`, `service.node.name`.
- `event.dataset` enables [log rate anomaly detection](/content/docs/solutions/observability/logs/inspect-log-anomalies/index.html) in the Elastic Observability app.

## Limitations and Considerations

The ecs-logging spec suggests that the first three fields in log records should be `@timestamp`, `log.level`, and `message`. As of version 1.5.0, this formatter does not follow this suggestion. It would be possible but would require creating a new Object in `ecsFields` for each log record. Given that ordering of ecs-logging fields is for human readability and does not affect interoperability, the decision was made to prefer performance.

## Reference

### `ecsFormat([options])`

- `options {type-object}` The following options are supported:
  - `convertErr {type-boolean}` Whether to convert a logged `err` field to ECS error fields. **Default:**`true`.
  - `convertReqRes {type-boolean}` Whether to log `req` and `res` HTTP request and response fields to ECS HTTP, User agent, and URL fields. **Default:**`false`.
  - `apmIntegration {type-boolean}` Whether to enable APM agent integration. **Default:**`true`.
  - `serviceName {type-string}` A "service.name" value. If specified this overrides any value from an active APM agent.
  - `serviceVersion {type-string}` A "service.version" value. If specified this overrides any value from an active APM agent.
  - `serviceEnvironment {type-string}` A "service.environment" value. If specified this overrides any value from an active APM agent.
  - `serviceNodeName {type-string}` A "service.node.name" value. If specified this overrides any value from an active APM agent.
  - `eventDataset {type-string}` A "event.dataset" value. If specified this overrides the default of using `${serviceVersion}`.

***