Winston: Proposed handling of info[SPLAT]

Created on 12 Jul 2018  ·  4Comments  ·  Source: winstonjs/winston

What's the feature?

This is a proposal for different handling of objects (JSON/array/other) for more versatile use cases.

In short, this proposal covers two things.

  1. Split out info.meta parsing from logform.splat
  2. Create a new transform that moves objects to options.meta

What problem is the feature intended to solve?

Various users (eg #1377, #1381, myself) have noted that treatment of objects/arrays is different in V3 from V2. In V2 these object types would be stored in the options.meta field and could be processed separately from the message and level and objects.

In V3, there is only one scenario/transform in which options.meta gets processed the same as in V2. This is when using the logform.splat transform. Splat will correctly parse meta only when a token is present in the log. EG:
logger.log('info', 'this is a string I want to splat %s', 'together', {an: 'object'})
will result in:
info: this is a string I want to splat together {"meta":{"an":"object"}} // with .simple() transform

If users don't want to use logform.splat, info.meta will not be processed. Instead, the meta objects will be left untouched in info[SPLAT].

Adding a logform that copies all info[SPLAT] to info[meta] will both meet expectations of V2 users and be more flexible. (Alternatively, directly adding the info[SPLAT] to the logger output and providing documentation on info[meta] => info[SPLAT] could be acceptable.)

A new transform (eg logform.captureAllMeta just for naming sake) could be as simple as:

// file captureAllMeta.js psuedo-code

module.exports = transform((info, opts) => {
  if (info[SPLAT]) {
    info.meta = info[SPLAT]
  return info;
});

The equivalent of 

format.combine(
        format.splat(),
        ...
    )

would become

format.combine(
        format.splat(),
        format.captureAllMeta(),
        ...
        // still need to have a formatter for options.meta.  maybe a separate feature request.
    )

The extraSplat code could be removed from splat() as it is handled in the subsequent (new) transform.

An added benefit is if users don't want to use format.splat(), they can still include format.captureAllMeta() as a standalone transform.

Is the absence of this feature blocking you or your team? If so, how?

Now knowing that info[SPLAT] stores what is missing in info[meta], this could be considered a knowledge/documentation gap.

Is this feature similar to an existing feature in another tool?

Extends V2 to V3 compatibility.

Is this a feature you're prepared to implement, with support from us?

Yes, I can do my best to support this.

To reproduce the problem

  • _winston version?_

    • [ ] winston@2

    • [X] winston@3

  • _node -v outputs:_ 6.10.0
  • _Operating System?_ (Windows, macOS, or Linux) Any
  • _Language?_ (all | TypeScript X.X | ES6/7 | ES5 | Dart) all

What is the problem?

Here is a sample of a V2 transport code with output:

        logger = winston.createLogger({
            transports: [
                new (winston.transports.Console)({
                    timestamp: function () {
                        return dateFormat(Date.now(), "HH:MM:ss.l");
                    },
                    formatter: function (options) {
                        // Return string will be passed to logger.
                        return options.timestamp() + ' ' + winston.config.colorize(options.level, options.level.toUpperCase()) + ' ' + (undefined !== options.message ? options.message : '') +
                            (options.meta && Object.keys(options.meta).length ? '\n\t' + JSON.stringify(options.meta) : '');
                    },
                    colorize: true,
                    level: container.settings.get('logLevel') || 'info',
                    stderrLevels: [],
                    handleExceptions: true,
                    humanReadableUnhandledException: true
                })
            ]
        });


// results of Winston@2
08:31:30.363 DEBUG Server starting complete.
        ["Server https started.","Server http started.","Server SSDP started.","Server MDNS started."]

In V3, I tried to replicate this with the following code, but the result did not display the array.

    const processObj = logform.format(info => {
             console.log('raw: ', raw)
            return info
        }
    )

    const myFormat = printf(info => {
        info.timestamp = dateFormat(Date.now(), "m/d/yy HH:MM:ss.l")
        return `${info.timestamp} ${info.level}: ${info.message}`;
    });

        logger = createLogger({
            level: container.settings.get('logLevel') || 'info',
            format: combine(
                processObj(),
                format.json(),
                timestamp({format: () => new Date().toLocaleString()}),
                format.colorize(),
                format.align(),
                format.splat(),
                myFormat
            ),
            transports: [new transports.Console()]
        });
        logger.exceptions.handle()
        logger.exitOnError = false;

results = [ 'Not starting https server.', 'Server http started.', 'Server SSDP started.', 'Server MDNS started.']
logger.debug('Server status', results)

// results of winston@3 -- missing the results array
6/25/18 08:16:30.501 debug:     Server starting complete.
investigate
Source
picture tagyoureit
👍3

Most helpful comment

So I just realized that using format.splat() does add a meta object to the info, BUT only if you also used interpolation. If you dont use interpolation, the meta is just mixed into the info object.

Also when using interpolation and meta, the format.simple() outputs the meta as {meta:{METAOBJ}} instead of just {METAOBJ} like v2.

picture tony-gutierrez on 7 Aug 2018
👍3

All 4 comments

I worked on this last night and created a branch with code that implements the above. Let me know if you want me to create a PR in Logform for this.

Here's a test file that I used...

'use strict';

const winston = require('winston');
const logform = require('logform')
const {SPLAT} = require('triple-beam');


const {createLogger, format, transports} = require('winston');
var logger = createLogger({
    format: format.combine(

        format.splat()
        , format.captureAllMeta()
        ,
        logform.format.printf(info => {

            // return `${info.timestamp} [${info.label}] ${info.level}: ${info.message}`;
            if ((!info.meta.length)) {
                return (`${info.level}: ${info.message}`);

            }
            else {
                return (`${info.level}: ${info.message}\n\t${JSON.stringify(info.meta)}`);

            }
        })
    ),
    transports: [new transports.Console()]
});


let count = 1

logger.log('info', count + '. this is a string I want to splat %s', 'together', {an: 'object'})
count += 1
console.log('---')

logger.log('info', count + '.this is a string followed by an object and an array', {an: 'object'}, ['an', 'array'])
count += 1
console.log('---')

logger.log('info', count + '.this is a string followed by an object and an array', ['an', 'array'], {an: 'object'})
count += 1
console.log('---')


logger.log('info', count + '. string followed by', 'a string')
console.log('---')
count += 1

logger.log('info', count + '. string followed by 2', 'separate', 'strings')
console.log('---')
count += 1

logger.log('info', count + '. string followed by', {an: 'object'})
console.log('---')
count += 1

logger.log('info', count + '. string followed by 2', {an: 'object'}, ['an', 'array'])
console.log('---')
count += 1

logger.log('info', count + '. string with token followed by string %s', 'string')
console.log('---')
count += 1

logger.log('info', count + '. string with token followed by obj %j', {idunno: 'notsure'})
console.log('---')
count += 1

logger.log('info', count + '. string with token followed by string %s and then objects', 'or here', {an: 'object'}, ['i', 'am', 'missed'])
console.log('---')
count += 1

logger.log('info', count + '. string with two tokens %s %s followed by one ', 'string')
console.log('---')
count += 1

logger.log('info', count + '. string with two tokens %s %j followed by one string and one object', 'string', {an: 'object'}, [1,2,3])
count += 1
console.log('---')


logger.info({message: 'just an object', [SPLAT]: {more:'stuff'}})
count += 1
console.log('---')

logger.info(['an','array'])
count += 1
console.log('---')

logger.info(['an','array',{with:'object'}])
count += 1
console.log('---')


logger.log({
    level: 'info',
    message: '%d: The answer to life, the universe and everything',
    splat: [42]
})
count += 1
console.log('---')

logger.log({
    level: 'info',
    message: '%d: The answer to life, the universe and everything',
    splat: [43],
    [Symbol.for('splat')]: [44]
})
count += 1
console.log('---')

logger.log({
    level: 'info',
    message: '%d: The answer to life, the universe and everything'
})
count += 1
console.log('---')

with the following outputs...

info: 1. this is a string I want to splat together
    [{"an":"object"}]
---
info: 2.this is a string followed by an object and an array
    [{"an":"object"},["an","array"]]
---
info: 3.this is a string followed by an object and an array
    [["an","array"],{"an":"object"}]
---
info: 4. string followed by
    ["a string"]
---
info: 5. string followed by 2
    ["separate","strings"]
---
info: 6. string followed by
    [{"an":"object"}]
---
info: 7. string followed by 2
    [{"an":"object"},["an","array"]]
---
info: 8. string with token followed by string string
---
info: 9. string with token followed by obj {"idunno":"notsure"}
---
info: 10. string with token followed by string or here and then objects
    [{"an":"object"},["i","am","missed"]]
---
info: 11. string with two tokens string %s followed by one
---
info: 12. string with two tokens string {"an":"object"} followed by one string and one object
    [[1,2,3]]
---
info: just an object
    [{"more":"stuff"}]
---
info: an,array
---
info: an,array,[object Object]  // expected... no %j token
---
info: 42: The answer to life, the universe and everything
---
info: 44: The answer to life, the universe and everything
---
info: %d: The answer to life, the universe and everything
---
tagyoureit picture tagyoureit on 13 Jul 2018

Added support to my code for info.splat in addition to info[SPLAT].

tagyoureit picture tagyoureit on 16 Jul 2018

Something like this is needed. Basically, if you cant upgrade to v3 without touching your existing log statements, AND be able to get meta as a SEPARATE object AFTER interpolation, then v3 is a non-starter for most exiting users. I have no idea why anyone thought it was a good idea to mix level, message, and meta into one big object. It is ripe for conflicts. info[Symbol.for('splat)] is also not reliable because it will mix interpolation values and meta together. We want meta SEPARATE.

tony-gutierrez picture tony-gutierrez on 7 Aug 2018
👍3

So I just realized that using format.splat() does add a meta object to the info, BUT only if you also used interpolation. If you dont use interpolation, the meta is just mixed into the info object.

Also when using interpolation and meta, the format.simple() outputs the meta as {meta:{METAOBJ}} instead of just {METAOBJ} like v2.

tony-gutierrez picture tony-gutierrez on 7 Aug 2018
👍3
Was this page helpful?
0 / 5 - 0 ratings

Related issues

greenhat616 picture greenhat616  ·  3Comments
Nepoxx picture Nepoxx  ·  4Comments
JaehyunLee-B2LiNK picture JaehyunLee-B2LiNK  ·  3Comments
ghost picture ghost  ·  4Comments
bertolo1988 picture bertolo1988  ·  3Comments