Node-redis: Streaming server response

Created on 14 Dec 2016  ·  6Comments  ·  Source: NodeRedis/node-redis

Hi, is possible with node_redis to stream the server response with commands like HMGET or KEYS?

Thanks.

Feature Request

All 6 comments

:+1: it would be great if we have implementation for it

@aletorrado I already had similar situation and I wanted to stream from HKEYS cmd
so I created a readable stream using HSCAN cmd, with this implementation:

const redis = require('redis');
const connectionParams = {};

class HKeysReadable extends require('stream').Readable {
  constructor(key, chunkSize, opt) {
    opt = opt || {};
    opt.objectMode = true;
    super(opt);
    this._cursor = 0;
    this.key = key;
    this.chunkSize = chunkSize;
    this.client = redis.createClient(connectionParams);
  }

  _read() {
    if (this._cursor === '0') {
      this.push(null);
      return this.client.quit();
    }

    this.client.hscan(this.key, this._cursor, 'COUNT', this.chunkSize, (err, res) => {
        if (err) {
          this.push(null);
          this.client.quit();
          return process.nextTick(() => this.emit('error', err));
        }

        this._cursor = res[0];

        /**
         *
         SCAN returns value as an array of two values, the first value is the new cursor to use in the next call,
         the second value is an array of elements like: [key1, val1, key2, val2, key3, val3, .....]
         In this stream, we need the keys only so we reduce that array to get only the odd values which contain the keys

         */
        let keys = res[1].reduce((accumulator, val, index)=> {
          if (!(index % 2)) {
            accumulator.push(val);
          }

          return accumulator;
        }, []);

        if (keys.length > 0) {
          this.push(keys);
        }
      }
    );
  }
};

@BridgeAR what do you think? is there a possibility to adapt similar implementation in Redis library as well?

Great @hossam-fares. Basically it may abstract cursors from node_redis clients, for a whitelisted set of redis commands (ie. SCAN, SSCAN, HSCAN, ZSCAN).

@hossam-fares I would definitely like to have something similar implemented :+1:

@BridgeAR cool, I will work on a PR for it

Has this been implemented?

Was this page helpful?
0 / 5 - 0 ratings

Related issues

Stono picture Stono  ·  6Comments

shmendo picture shmendo  ·  6Comments

jackycchen picture jackycchen  ·  4Comments

Atala picture Atala  ·  3Comments

Mickael-van-der-Beek picture Mickael-van-der-Beek  ·  6Comments