Mopidy: Look into adding a gstreamer fifo sink

Created on 8 Jul 2014  ·  73Comments  ·  Source: mopidy/mopidy

This keeps coming up as people really seem to like ncmpcpp's visualizer

Trick for making a reliable FIFO sink is twofold. You need to use non-blocking writes, which means opening the write socket will fail unless there is a reader, hence you need a reader as well. Secondly if the buffer fills up because the external reader falls behind or there is none, your internal reader needs to clear the buffer.

Following code captures the basic gist of this, but still needs some more work with respect to error handling.

import errno
import os
import stat

LINUX_FIFO_BUFFER_SIZE = 65536


class FifoStreamer(object):                                                     
    def __init__(self, location):                                               
        self.location = location                                                
        self.reader = None                                                      
        self.writer = None                                                      

    def create(self):                                                           
        try:                                                                    
            mode = os.stat(self.location).st_mode                               
            if not stat.S_ISFIFO(mode):                                         
                raise Exception('File exists but is not a FIFO')                
        except OSError as e:                                                    
            if e.errno == errno.ENOENT:                                         
                os.mkfifo(self.location)                                        
            else:                                                               
                raise                                                           

        # TODO: wrap in could not open reader / writer?
        self.reader = os.open(self.location, os.O_NONBLOCK | os.O_RDONLY)       
        self.writer = os.open(self.location, os.O_NONBLOCK | os.O_WRONLY)       

    def close(self):                                                            
        # TODO: make closing robust
        os.close(self.writer)                                                   
        os.close(self.reader)                                                   

    def write(self, data):                                                      
        while data:                                                             
            try:                                                                
                written = os.write(self.writer, data)                           
                data = data[written:]                                           
            except OSError as e:                                                
                if e.errno == errno.EINTR:                                      
                    continue                                                    
                elif e.errno in (errno.EAGAIN, errno.EWOULDBLOCK):              
                    self.flush()                                                
                else:                                                           
                    raise                                                       

    def flush(self):                                                            
        while True:                                                             
            try:                                                                    
                if not os.read(self.reader, LINUX_FIFO_BUFFER_SIZE):                             
                    break                                                       
            except OSError as e:                                                
                if e.errno in (errno.EAGAIN, errno.EWOULDBLOCK):                
                    break                                                       
                elif e.errno == errno.EINTR:                                    
                    continue                                                    
                else:                                                           
                    raise 

The code above now needs to be integrated with GStreamer do actually do it's thing. Assuming 0.10 this is actually very much doable, reason I am hesitant is that we are planing a move to 1.x and the gir binding are not suited for making elements in python. In the case of this code in 1.x the problem boils down to not being able to create the pad templates that BaseSink expect to find. Creating this as a plain element might be doable, but you would need to re-implement way to much of the handling BaseSink makes sure you get right.

import gobject                                                                  

import pygst                                                                    
pygst.require('0.10')                                                           
import gst                                                             


class FifoSink(gst.BaseSink):                                                   
    __gstdetails__ = (                                                          
        'FifoSink',                                                             
        'Sink',                                                                 
        'Sink designed to handle FIFO output.',                        
        'Mopidy')                                                               

    __gsttemplates__ = (gst.PadTemplate('sink', gst.PAD_SINK, gst.PAD_ALWAYS,   
                                        gst.caps_new_any()),)                   

    # TODO: don't allow changing location in flight, i.e. create getter/setter
    location = gobject.property(type=str)                                       

    def __init__(self):                                                         
        gst.BaseSink.__init__(self)                                             
        self.streamer = None                                                    

    def do_start(self):                                                                                               
        self.streamer = FifoStreamer(self.location)                             
        self.streamer.create()                                                  
        return True                                                             

    def do_stop(self):                                                          
        self.streamer.close()                                                   
        return True                                                             

    def do_render(self, buf):                                                   
        try:                                                                    
            self.streamer.write(bytes(buf))                                     
            return gst.FLOW_OK                                                  
        except OSError as e:                                                    
            self.error("Failed: %s", e)                                         
            return gst.FLOW_ERROR                                               


gobject.type_register(FifoSink)                                                 
gst.element_register(                                                           
    FifoSink, 'fifosink', gst.RANK_MARGINAL)                                    

if __name__ == '__main__':                                                      
    import gobject                                                              
    gobject.threads_init()                                                      

    output = """                                                                
capsfilter caps="audio/x-raw-int,width=16,rate=44100,channels=1" ! 
tee name=t 
t. ! queue ! alsasink                                                           
t. ! queue ! fifosink location=/tmp/test2.fifo                                  
"""                                                                             

    sink = gst.parse_bin_from_description(                                      
        output, ghost_unconnected_pads=True)                                    

    playbin = gst.element_factory_make('playbin2')                              
    playbin.set_property('audio_sink', sink) 

Note that one problem I ran into testing this was actually forgetting to match the audio format expected by ncmpcpp, so make sure mono/stereo do indeed match up.

C-enhancement A-audio

Most helpful comment

FYI I added support for gstreamer's udpsink to ncmpcpp so now it can be used directly without fifo hacks.

See https://github.com/ncmpcpp/ncmpcpp/commit/fb886f687014e22b2fe1477da855be5201063ea8 for more details.

Below a teaser. This is mopidy + mopidy-spotify + mopidy-mpd + ncmpcpp.

https://user-images.githubusercontent.com/387658/102549720-e41bf980-40bc-11eb-8127-70889011e52f.mp4

All 73 comments

Where did this go? will this be added to the main package?

This isn't being worked on as we would need to convert it to GStreamer 1.0, and all my experiments with that so far have uncovered problems when doing custom elements in python. There are other ways of maybe solving it all, but not something I have time to work on.

I'd like to see FIFO support added :)

I just found this searching for a way to get ncmpcpp's music visualizer to work with mopidy. I am in favor.

Yes, that's the same reason why I found this. It would be really great to
be able to visualize streamed music,

2014-11-06 2:19 GMT-05:00 Blake [email protected]:

I just found this searching for a way to get ncmpcpp's music visualizer to
work with mopidy. I am in favor.


Reply to this email directly or view it on GitHub
https://github.com/mopidy/mopidy/issues/775#issuecomment-61936376.

There are gstreamer visualisation plugins, can you not just use those?
On 6 Nov 2014 08:35, "Diego Berrocal" [email protected] wrote:

Yes, that's the same reason why I found this. It would be really great to
be able to visualize streamed music,

2014-11-06 2:19 GMT-05:00 Blake [email protected]:

I just found this searching for a way to get ncmpcpp's music visualizer
to
work with mopidy. I am in favor.


Reply to this email directly or view it on GitHub
https://github.com/mopidy/mopidy/issues/775#issuecomment-61936376.


Reply to this email directly or view it on GitHub
https://github.com/mopidy/mopidy/issues/775#issuecomment-62007426.

Note that those are on their way out in the next release.

Is there someone working on this? Would be really nice to be able to use the ncmpcpp visualizer with mopidy.

Not that I know of, and we likely don't want to do this as GStreamer element as it would block our migration to GStreamer 1.x.

This is IMO also blocked on adding back proper support for outputs.

Just adding that I also came across this issue when trying to add visualizer support to Mopidy/ncmpcpp. Would love to have this in the future even if it's not being worked on currently. I'd also like to extend my utmost respect for those who contribute their time to open source projects. :smile:

Ideally this would work remotely. Mopidy's power lies really in being a solution that can be in your private cloud.

In general, FIFOs are not a very portable method of interprocess communications and should be avoided IMO. They also have some nasty side-effects, already mentioned in this thread. I think you are better off dumping your packets out using a UDP sink personally....that's a more generic solution and can easily be adapted to something that writes to a FIFO (if necessary) outside of Mopidy....it's a single line of shell script to read data from a UDP port (using netcat) and write it to a FIFO.

That sounds good. May be clients lie ncmpcpp would also allow using that as input.

Another idea is to use PulseAudio directly as it would require no extra implementation on Mopidy's side. The Docker container I made uses TCP to access PulseAudio over network. ncmpcpp could actually use that directly.

From what I've read, the visualizer is just running an FFT on blocks of samples which are read from a FIFO file. The point I am making is that you can create a FIFO outside of Mopidy using mkfifo and then read the UDP data from Mopidy using netcat and redirect the output from netcat to the FIFO file. Thus nothing needs to change as far as ncmpcpp is concerned, it just reads the FIFO file.

To achieve what you're aiming on Mopidy requires no implementation effort since you can change the audio output property to output to a udpsink GStreamer element. More likely you would want to arrange it as part of a "tee" which outputs to both your ALSA/Pulse sink and the udpsink.

The following should work....not tested so might need some tweeking.

In Mopidy audio configuration:

output = tee name=t ! queue ! autoaudiosink t. ! queue ! udpsink port=5555

On your Linux host, launch netcat to listen on localhost for UDP data on port 5555 and output the samples to the FIFO:

mkfifo /tmp/mopidy.fifo
nc -u -l 127.0.0.1 5555 > /tmp/mopidy.fifo &

In your mpd.conf:

audio_output {
    type                    "fifo"
    name                    "my_fifo"
    path                    "/tmp/mopidy.fifo"
    format                  "44100:16:2"
}

Note: If you are using a PulseAudio output, it might be possible to do something similar with its TCP direction connection module. However, you might need to reverse engineer any protocol it runs over the top of the TCP connection e.g., it might not be possible to treat it as a raw sink.

Thank @liamw9534 for sharing. It would have taken me a while to understand the config enough to do that or even know I could. I'll try to add that to the Docker container see if it works.

This might run into issues if you don't have something emptying the fifo though. As netcat will fill the kernel buffer and then fail. And depending on how the FIFO is opened you might also run into issues as FIFOs can require a reader to be present to be writable :/

But you could adapt my FifoStreamer code to accept the UDP data and handle this for you as I hopefully already have handled most of the error cases you can run into there.

@adamcik If you have a working script I'd be glad to include it as part of docker-ncmpcpp for people to use. I'd also change docker-mopidy to broadcast on UDP by default using the output trick above.

I did some basic experiments on the command-line and it all works fine provided you use a connection timeout e.g., nc -kluw 1 127.0.0.1 5555 > /tmp/mopidy.fifo. This forces netcat to always listen for a new source port connection after the last connection drops (1s timeout).

The above command works irrespective of whether or not anything is reading from the FIFO file and also works if the process reading from the FIFO file stops and closes the file.

@wernight see the description of this bug, that is all the code I have.

And just for belts and braces, I would stick the command in a while loop e.g., while :; do nc -kluw 1 127.0.0.1 5555> /tmp/mopidy.fifo; sleep 1; done - if it does exit, it will just launch again after 1 second delay.

@liamw9534 You have a mpd.conf there. I am confused by this. You are running a standalone MPD server in addition to mopidy-mpd?

@lrvick yes my mistake, was of course thinking about config that needs to be added to ~/.ncmpcpp/config

.config/mopidy/mopidy.conf

...

[audio]
output = tee name=t ! queue ! autoaudiosink t. ! queue ! udpsink port=5555
$ mkfifo /tmp/mopidy.fifo
$ while :; do nc -kluw 1 127.0.0.1 -p 5555> /tmp/mopidy.fifo; sleep 1; done

(added -p here).

~/.ncmpcpp/config

audio_output {
    type                    "fifo"
    name                    "my_fifo"
    path                    "/tmp/mopidy.fifo"
    format                  "44100:16:2"
}

(done also in wernight/docker-mopidy and wernight/docker-ncmpcpp)

However it seems it cannot connect to 5555 even though the log of Mopidy shows it understood the output config. netstat shows also no open UDP on 5555. I don't fully understand the output line: tee is a bash command, the rest looks like Mopidy stuff.

I did however find udp6 [::]:51307 while playing music.

The output is interpreted as a GstBin element that gets passed to playbin2 as part of Mopidy - under the hood this is just a Gstreamer pipeline, nothing special really. The tee is actually therefore a Gstreamer element. It has nothing to do with the shell command tee.

So you can approximate what is happening inside Mopidy on the command-line doing something like:

gst-launch audiotestsrc ! tee name=t ! queue ! autoaudiosink t. ! queue ! udpsink port=5555 &

This will start playing a sinusoidal tone. Then run the following command:

nc -kluw 1 127.0.0.1 5555

I can then start to see lots of strange characters on my display....this is the test audio stream outputting to stdout from nc. This effectively proves the underlying approach works. Can you confirm you can replicate the same behaviour?

You can then try to add the FIFO in as a second step and confirm you can cat the FIFO to stdout and see the same strange characters.

I would also double check the configuration format needed by ncmpcpp. I had a brief look at https://wiki.archlinux.org/index.php/Ncmpcpp#Enabling_visualization and can see that maybe the config format needed is different e.g.,

visualizer_fifo_path = "/tmp/mpd.fifo"
visualizer_output_name = "my_fifo"
visualizer_sync_interval = "30" 
visualizer_in_stereo = "yes"
visualizer_type = "spectrum" (spectrum/wave)

I don't use ncmpcpp, so the problem could be here too...

$ apt-get install gstreamer-tools
$ gst-launch audiotestsrc ! tee name=t ! queue ! autoaudiosink t. ! queue ! udpsink port=5555 &
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstPulseSinkClock
$ nc -kluw 1 127.0.0.1 -p 5555
no connection : Connection timed out

(with nc ... -p else I get an error)

I do hear the tone playing though.

What distro are you using? Also when you type nc -h what is the output?

$ nc -h
OpenBSD netcat (Debian patchlevel 1.89-4ubuntu1)
This is nc from the netcat-openbsd package. An alternative nc is available
in the netcat-traditional package.
usage: nc [-46DdhklnrStUuvzC] [-i interval] [-P proxy_username] [-p source_port]
      [-s source_ip_address] [-T ToS] [-w timeout] [-X proxy_protocol]
      [-x proxy_address[:port]] [hostname] [port[s]]
    Command Summary:
        -4      Use IPv4
        -6      Use IPv6
        -D      Enable the debug socket option
        -d      Detach from stdin
        -h      This help text
        -i secs     Delay interval for lines sent, ports scanned
        -k      Keep inbound sockets open for multiple connects
        -l      Listen mode, for inbound connects
        -n      Suppress name/port resolutions
        -P proxyuser    Username for proxy authentication
        -p port     Specify local port for remote connects
        -q secs     quit after EOF on stdin and delay of secs
        -r      Randomize remote ports
        -S      Enable the TCP MD5 signature option
        -s addr     Local source address
        -T ToS      Set IP Type of Service
        -C      Send CRLF as line-ending
        -t      Answer TELNET negotiation
        -U      Use UNIX domain socket
        -u      UDP mode
        -Z      DCCP mode
        -v      Verbose
        -w secs     Timeout for connects and final net reads
        -X proto    Proxy protocol: "4", "5" (SOCKS) or "connect"
        -x addr[:port]  Specify proxy address and port
        -z      Zero-I/O mode [used for scanning]
    Port numbers can be individual or ranges: lo-hi [inclusive]

The -p option is not needed with the nc I am using on Ubuntu. The man page also implies that it should be an error to specify -p in conjunction with -l:

     -l      Used to specify that nc should listen for an incoming connection
             rather than initiate a connection to a remote host.  It is an error
             to use this option in conjunction with the -p, -s, or -z options.
             Additionally, any timeouts specified with the -w option are ignored.

Using Debian Wheezy.

Here are full steps for Ubuntu starting by installing Docker:

  1. Install Docker:

$ wget -qO- https://get.docker.com/ | sh $ sudo gpasswd -a ${USER} docker

  1. Enable remote PulseAudio
  2. Run on Debian:

$ docker run --rm -it \ -e PULSE_SERVER=tcp:$(hostname -i):4713 \ -e PULSE_COOKIE_DATA=$(pax11publish -d | grep --color=never -Po '(?<=^Cookie: ).*') \ debian:wheezy $ apt-get update && apt-get install -y netcat $ nc -h [v1.10-40] connect to somewhere: nc [-options] hostname port[s] [ports] ... listen for inbound: nc -l -p port [-options] [hostname] [port] options: -c shell commands as `-e'; use /bin/sh to exec [dangerous!!] -e filename program to exec after connect [dangerous!!] -b allow broadcasts -g gateway source-routing hop point[s], up to 8 -G num source-routing pointer: 4, 8, 12, ... -h this cruft -i secs delay interval for lines sent, ports scanned -k set keepalive option on socket -l listen mode, for inbound connects -n numeric-only IP addresses, no DNS -o file hex dump of traffic -p port local port number -r randomize local and remote ports -q secs quit after EOF on stdin and delay of secs -s addr local source address -T tos set Type Of Service -t answer TELNET negotiation -u UDP mode -v verbose [use twice to be more verbose] -w secs timeout for connects and final net reads -z zero-I/O mode [used for scanning] port numbers can be individual or ranges: lo-hi [inclusive]; hyphens in port names must be backslash escaped (e.g. 'ftp\-data'). $ nc -kluw 1 127.0.0.1 5555 UDP listen needs -p arg $ apt-get install -y net-tools gstreamer-tools gstreamer0.10-plugins-good $ echo -ne $(echo $PULSE_COOKIE_DATA | sed -e 's/../\\x&/g') >$HOME/pulse.cookie $ export PULSE_COOKIE=$HOME/pulse.cookie $ gst-launch audiotestsrc ! tee name=t ! queue ! autoaudiosink t. ! queue ! udpsink port=5555 & $ nc -kluw 1 127.0.0.1 -p 5555 no connection : Connection timed out

Tried with netcat-openbsd package which gives:

$ nc -h
OpenBSD netcat (Debian patchlevel 1.105-7)
...

Now nc just shows nothing while the tone is playing (which seems somewhat better):

$ nc -kluw 1 127.0.0.1 5555
(nothing here)

Ok, open up two terminals and in the first terminal run:

nc -kluw 1 127.0.0.1 5555

Then in the second terminal run:

nc -u 127.0.0.1 5555

Enter some characters on stdin in the second terminal and press ENTER. Do you see the same characters appear in the first terminal on stdout?

Yes that works. Even got it working cross Dockers:

$ docker run --rm --name nc1 -it my-debian-with-nc nc -kluw 1 0.0.0.0 5555
Hello

$ docker run --rm --link nc1:nc1 -it my-debian-with-nc nc -u nc1 5555
Hello

So this doesn't explain why nc showed nothing with gst-launch. However I found out that I would need to set host=SOMETHING for it to work across containers. This will be problematic. Seems it cannot broadcast. But that's something I'll can solve once it works at least on a single container.

Aha! If I set host=0.0.0.0 it works!

Do you need to send the data over a network? I had assumed you only needed the packets on the local machine. Using udpsink host=0.0.0.0 port=5555 will broadcast packets everywhere on your network. Probably not what you want...

You should be able to say host=127.0.0.1 instead to restrict packets to local loopback.

I suspect the real issue is therefore down to the default routing configuration. By default, on your system, the outgoing packets are channeled to a different interface. Possibly different versions of the gstreamer element or differences in ip routing setup.

From the container I had to use 0.0.0.0 or possible the IP of the machine. Default setting wasn't working even on a single machine/container.

But yes I have to use network now because it's good practice to keep ncmpcpp and mopidy in separate Docker containers, which means they are on different networks. That's really good for safety and isolation and such, but it creates a bit trouble:

ncmpcpp client connects to MPD server to ask for music streaming, and only after that point the IP of ncmpcpp is known and UDP packet should be broadcast to its IP; but currently the IP is hardcoded in Mopidy configuration.

I don't have a solution right now. I'm thinking of solutions:

  • Creating the FIFO in Mopidy and sharing it with ncmpcpp, but I'm afraid nothing will clear the FIFO and that's big trouble. Also sharing files is not as clean as network sockets.
  • Local daemon monitoring connections and forwarding UDP traffic to found client IP; hack.
  • UDP SSH tunnel, but that's complex, adds useless encryption, adds another daemon... bad!
  • Looking at code to see if udpsink host can be defaulted to the first client's IP or even better to all connected clients' IPs
  • Tell Docker to share host via --net=container:NAME_or_ID which adds another parameter for users to type, but that would simplify a lot!

There are solutions for this circular dependency. Still feel like a horrible hack but it should be easy enough for users to use in the end. Thanks liamw9534 for helping especially with the nc commands which I was alien with.

The FIFO is working except that There is no output named "my_fifo"!. It seems that the name has to match the one in mdp.conf:

 audio_output {
        type            "fifo"
        name            "my_fifo"
        path            "/tmp/mpd.fifo"
        format           "44100:16:2"
}

I may try to share the FIFO file if I know how to add this audio output to Mopidy. nausea (a simple visuallizer-ony) seem to have same requirements.

I found this documentation of visualizer_output_name:

  • Below parameter is needed for ncmpcpp to determine which output provides data for visualizer and thus allow syncing between visualization and sound as currently there are some problems with it.
  • fifo output, whose format parameter has to be set to 44100:16:1 for mono visualization or 44100:16:2 for stereo visualization

I am not sure what you mean by saying "on different networks". If you want to isolate the audio traffic to the local machine then you should use 127.0.0.1 for udpsink. If you choose 0.0.0.0 then it will direct packets using the default route which likely means sending UDP packets over the network....in general, I'd say this is a bad idea because the network traffic will consume a lot of bandwidth i.e., 2 x 16 x 44100 = 1.41 Mbps.

In theory, it should be possible to configure nc with 0.0.0.0 irrespective of which network interface is used by udpsink. This will listen on all network interfaces. That's fine as long as you don't use the same IP port number for other services on different network interfaces.

I used my last suggestion and put both on the same network. Those networks are local anyway, it's just a logical separation. That works to build the FIFO, just ncmpcpp doesn't want it (see my previous response, 2 above).

Any idea about that channel name? Or if there is another way to a FIFO like mpd.conf? Doesn't seem to work at all without.

I thought mpd.conf was used for the MPD daemon configuration only. In this case, that should be redundant because mopidy is the MPD daemon and your docker is creating the FIFO file. In which case, the only thing that should be necessary is to ensure the .ncmpcpp/config is matching for the client, right?

Kind of. What exactly it's doing I don't know. But as you can see the visualizer_output_name = "my_fifo" must match audio_output { name "my_fifo" }. Currently ncmpcpp shows There is no output named "my_fifo". Description of visualizer_output_name is just 5 posts above. It looks like it's actually connecting directly somehow to the output to sync.

I don't use any of this but it looks to me like the visualizer_output_name should match one of your MPD (protocol, not program) outputs (i.e. something in mpc outputs). Mopidy doesn't currently allow for configurable outputs and only has one MPD output: "Mute".

You would probably want to unset visualizer_output_name, if possible.

Tried not setting it. Still no visualizer visible. I wonder about the output format of udpsink: Should be 44100:16:2 for visualizers to work (ncmpcpp or nausea).

Tried Mute just in case, got a very funny experience: It just muted the sound, without showing an error but also without showing visualizers. Or more exactly it disabled the Mute output.

I do wonder about where the FFT function is performed for the visualizer.
Is it performed at the output of the FIFO or at the input to the FIFO? You
would need to check the MPD client code to see how it handles the FIFO data.

Is there a way to support either, just for testing, even if it would normally create a monstruously huge file.

You don't need to support either. You need to find out where the FFT is
supposed to be done and then choose the correct approach. If the FFT is
done in the visualizer then it would just be a complete waste of effort to
implement it again.

On 13 May 2015 at 15:09, Werner Beroux [email protected] wrote:

Is there a way to support either, just for testing, even if it would
normally create a monstruously huge file.


Reply to this email directly or view it on GitHub
https://github.com/mopidy/mopidy/issues/775#issuecomment-101676033.

FFT is done in by the visualizer and it requires the specific audio format in the FIFO to work for that. Reference: http://git.2f30.org/nausea/about/

Good - then there is no reason for it not to work! Have you done a
side-by-side comparison of a system running a standard MPD daemon with a
local ncmpcpp client? What happens if you rename the FIFO filename in both
the configuration files? Does it still work? Just trying to see if a
hardcoded path exists somewhere...

On 13 May 2015 at 15:57, Werner Beroux [email protected] wrote:

FFT is done in by the visualizer and it requires the specific audio format
in the FIFO to work for that. Reference: http://git.2f30.org/nausea/about/


Reply to this email directly or view it on GitHub
https://github.com/mopidy/mopidy/issues/775#issuecomment-101702624.

My comment was based off the ncmpcpp source.

Mopidy's Mute output is disabled by default i.e. the audio is not muted. The ncmpcpp visualiser will take the output you gave it, disable it (no effect) and then enable it, thus muting the audio. Arguably they should just toggle it twice.

The audio being in the wrong format is a good possibility. You have nothing in your gstreamer pipeline to ensure it is correct unlike @adamcik had in his code above.

Hard to see how the format could be a problem....it's 16-bit signed
integers. The worst that could happen is that you swap the L/R channels?

On 13 May 2015 at 16:33, Nick Steel [email protected] wrote:

My comment was based off the ncmpcpp source
http://git.musicpd.org/cgit/mirror/ncmpcpp.git/tree/src/visualizer.cpp.

Mopidy's Mute output is disabled by default i.e. the audio is not muted.
The ncmpcpp visualiser will take the output you gave it, disable it (no
effect) and then enable it, thus muting the audio. Arguably they should
just toggle it twice.

The audio being in the wrong format is a good possibility. You have
nothing in your gstreamer pipeline to ensure it is correct unlike @adamcik
https://github.com/adamcik had in his code above.


Reply to this email directly or view it on GitHub
https://github.com/mopidy/mopidy/issues/775#issuecomment-101716947.

And even if the byte ordering was wrong, you might then just expect the FFT
output to generate the wrong frequency response......but you'd still see
something.

On 13 May 2015 at 16:37, Liam Wickins [email protected] wrote:

Hard to see how the format could be a problem....it's 16-bit signed
integers. The worst that could happen is that you swap the L/R channels?

On 13 May 2015 at 16:33, Nick Steel [email protected] wrote:

My comment was based off the ncmpcpp source
http://git.musicpd.org/cgit/mirror/ncmpcpp.git/tree/src/visualizer.cpp.

Mopidy's Mute output is disabled by default i.e. the audio is not muted.
The ncmpcpp visualiser will take the output you gave it, disable it (no
effect) and then enable it, thus muting the audio. Arguably they should
just toggle it twice.

The audio being in the wrong format is a good possibility. You have
nothing in your gstreamer pipeline to ensure it is correct unlike
@adamcik https://github.com/adamcik had in his code above.


Reply to this email directly or view it on GitHub
https://github.com/mopidy/mopidy/issues/775#issuecomment-101716947.

yeh true, so maybe it's because leaving Mopidy's Mute output enabled means the udpsink won't send any data.

cat /tmp/mdp.fifo (created from UDP as above) correctly shows constant stream of data.

I looked at visualizer ncmpcpp code:

  1. opens the file as O_RDONLY | O_NONBLOCK
  2. reads from top; given it's set for stereo in my mopidy config, it should be 44100:16:2.

Anyone can also try from my branch (after having installed Docker):

git clone https://github.com/wernight/docker-mopidy.git
cd docker-mopidy
git checkout udpsink
docker build -t mopidy .
cd ..

git clone https://github.com/wernight/docker-ncmpcpp.git
cd docker-ncmpcpp
git checkout udpsink
docker build -t ncmpcpp .
cd ..

# Remember to enable PulseAudio over network 

docker run --name mopidy -d \
      -e PULSE_SERVER=tcp:$(hostname -i):4713 \
      -e PULSE_COOKIE_DATA=$(pax11publish -d | grep --color=never -Po '(?<=^Cookie: ).*') \
      mopidy
docker run --rm -it --link mopidy:mopidy ncmpcpp --host mopidy

Had to spend quite some time wrangling netcat as -k didn't work for me, but here is what I came up with to autorestart it on song switches:

while :; do yes $'\n' | nc -lu 127.0.0.1 5555 > /tmp/mopidy.fifo; done

other options as above basically.

@S0lll0s @wernight netcat -k also didn't work for me and had to be restarted after song changes. While your solution works, I've found socat to work better:

while :; do socat -d -d -T 1 -u UDP4-LISTEN:5555 OPEN:/tmp/mopidy.fifo; done

For arch linux users:

To get nc with -k support you need to install the openbsd-netcat package, but it didn't work for me either, so try using @SjRNMzU 's script - you need the socat package to run it.

@johnhamelink (I presume you are running Arch from your comment) Have you had issues with the visualizer trying to "catch up" after you pause a song using the socat solution?

@theos-space I can't say I've noticed that myself

I'm on Arch and couldn't get it to work with either nc (both gnu and openbsd) or socat. Additionally, I can't read from gst-launch-1.0 ... udpsink port=5555 with nc -kluw 1 127.0.0.1 5555, as it prints nothing to the screen.

Can confirm this works on debian buster (4.11.6-1), mopidy 2.1.0-1, gstreamer1.0-tools 1.12.2-1, ncmpcpp 0.7.4-1+b3, socat 1.7.3.2-1:

start socat in the system startup:

open_mpd_fifo() {
    local fifo
    readonly fifo='/tmp/mpd.fifo'

    mkfifo "$fifo"
    while :; do socat -d -d -T 1 -u UDP4-LISTEN:5555 OPEN:"$fifo"; done
}

mopidy [audio] conf (note host had to be defined for udpsink):

[audio]
output = tee name=t ! queue ! autoaudiosink t. ! queue ! udpsink host=127.0.0.1 port=5555

ranisalt for Arch you have to compile ncmpcpp with fifo support check here: https://bbs.archlinux.org/viewtopic.php?id=99915

Here's my working Arch setup, after reading the rest of the thread + docs + manpages + assorted blogs. I'm running Mopidy as a system service and sending the audio to the user service PulseAudio via TCP. I've included some additional annotations that will hopefully help the less experienced.

Packages

All of these are from the official Arch repos:

gstreamer 1.12.3-1
mopidy 2.1.0
ncmpcpp 0.8.1  # this had fifo support without any special compiling for me
openbsd-netcat 1.178_3-1  # gnu-netcat lacks the -k flag, as discussed above
pulseaudio 11.1

Setup

Enable mopidy system service:

sudo systemctl enable mopidy.service

Replace mopidy's system config with a symlink to the user config for ease of editing (optional):

sudo ln -sf ~/.config/mopidy/mopidy.conf /etc/mopidy/mopidy.conf

Create the fifo file:

mkfifo "/tmp/mpd.fifo"

Configs

in ~/.config/mopidy/mopidy.conf (or /etc/mopidy/mopidy.conf if you didn't create the symlink): Tell mopidy to send audio to a pulseaudio server listening on localhost and to a UDP sink at localhost:5555

[audio]
output = tee name=t ! queue ! pulsesink server=127.0.0.1 t. ! queue ! udpsink host=127.0.0.1 port=5555

in ~/.ncmpcpp/config: Tell ncmpcpp where to find the fifo file it needs to listen to, and some other miscellaneous settings

visualizer_fifo_path = "/tmp/mpd.fifo"
visualizer_output_name = "my_fifo"
visualizer_sync_interval = "30"
visualizer_in_stereo = "yes"
visualizer_type = "spectrum"
visualizer_look = "+|"

in ~/.config/pulse/default.pa: Tell pulseaudio to accept audio over TCP from localhost (the setting is likely already there and can just be uncommented)

load-module module-native-protocol-tcp auth-ip-acl=127.0.0.1

in ~/.zshrc (should also work in ~/.bashrc): Create a wrapper function "nplayer" to handle starting and stopping netcat with ncmpcpp:

nplayer () (nc -kluw 1 127.0.0.1 5555 > /tmp/mpd.fifo & trap "kill $!" EXIT; ncmpcpp)

What this does:

  • define a function with a subshell wrapping its whole contents
  • start a netcat process in the background to listen for the audio data sent to the UDP sink and redirect it to the fifo created earlier. Flags:

    • -k keep listening after current connection is completed

    • -l enable listening mode

    • -u enable UDP mode

    • -w NUM timeout idle connections after NUM seconds (1 sec in this case)

  • set an EXIT trap to kill the netcat process when the ncmpcpp process terminates and causes the function subshell to exit
  • start a ncmpcpp process in the foreground

Im running mopidy on Manjaro (arch distro). I was having trouble with getting any output through socat or netcat. I was able to observer the packets coming in with tcpdump though:
sudo tcpdump -i lo -n udp port 5555 -XX

It took me a long while and I wouldn't have found the issue without @mosbasik .
The issue was not having host=127.0.0.1 after udpsink, why this works for other people and not me, I dont know. But if you're not getting any output it possible this is the case. These 2 worked for me:

[audio]
output = tee name=t ! queue ! autoaudiosink server=127.0.0.1 t. ! queue ! udpsink host=127.0.0.1 port=5555↪

or

[audio]
output = tee name=t ! queue ! autoaudiosink  t. ! queue ! udpsink host=127.0.0.1 port=5555↪

On a side note: I wasn't able to get pulsesink to work so I had to leave it at autoaudiosink, which is what I had it set too before. However the visualizer seems to be really really slow for some reason, Im not sure if that is because of autoaudiosink Works Perfect after a Reboot .. still slightly slow though. With pulselink it kept complaining about gstreamer plugins. Even after (I assume) I installed every plugin package for gst on the official Arch repos. Note flump3dec and mad


Output of mopidy deps

Executable: /usr/bin/mopidy
Platform: Linux-4.16.7-1-MANJARO-x86_64-with-glibc2.2.5
Python: CPython 2.7.15 from /usr/lib/python2.7
Mopidy: 2.2.1 from /usr/lib/python2.7/site-packages
  Pykka>=1.1: 1.2.1 from /usr/lib/python2.7/site-packages
  requests>=2.0: 2.20.1 from /usr/lib/python2.7/site-packages
    chardet>=3.0.2: 3.0.4 from /usr/lib/python2.7/site-packages
    idna>=2.5: 2.7 from /usr/lib/python2.7/site-packages
    urllib3>=1.21.1: 1.24.1 from /usr/lib/python2.7/site-packages
  setuptools: 40.5.0 from /usr/lib/python2.7/site-packages
  tornado>=4.4: 5.1.1 from /usr/lib/python2.7/site-packages
    futures: 3.2.0 from /usr/lib/python2.7/site-packages
    singledispatch: 3.4.0.3 from /usr/lib/python2.7/site-packages
      six: 1.11.0 from /usr/lib/python2.7/site-packages
    backports_abc>=0.4: 0.5 from /usr/lib/python2.7/site-packages
Mopidy-Iris: 3.4.9 from /usr/lib/python2.7/site-packages
  setuptools>=3.3: 40.5.0 from /usr/lib/python2.7/site-packages
  pylast>=1.6.0: 2.3.0 from /usr/lib/python2.7/site-packages
    six: 1.11.0 from /usr/lib/python2.7/site-packages
  Mopidy>=2.0: 2.2.1 from /usr/lib/python2.7/site-packages
    Pykka>=1.1: 1.2.1 from /usr/lib/python2.7/site-packages
    requests>=2.0: 2.20.1 from /usr/lib/python2.7/site-packages
      chardet>=3.0.2: 3.0.4 from /usr/lib/python2.7/site-packages
      idna>=2.5: 2.7 from /usr/lib/python2.7/site-packages
      urllib3>=1.21.1: 1.24.1 from /usr/lib/python2.7/site-packages
    setuptools: 40.5.0 from /usr/lib/python2.7/site-packages
    tornado>=4.4: 5.1.1 from /usr/lib/python2.7/site-packages
      futures: 3.2.0 from /usr/lib/python2.7/site-packages
      singledispatch: 3.4.0.3 from /usr/lib/python2.7/site-packages
        six: 1.11.0 from /usr/lib/python2.7/site-packages
      backports_abc>=0.4: 0.5 from /usr/lib/python2.7/site-packages
  Mopidy-Local-Images>=1.0: 1.0.0 from /usr/lib/python2.7/site-packages
    setuptools: 40.5.0 from /usr/lib/python2.7/site-packages
    Mopidy>=1.1: 2.2.1 from /usr/lib/python2.7/site-packages
      Pykka>=1.1: 1.2.1 from /usr/lib/python2.7/site-packages
      requests>=2.0: 2.20.1 from /usr/lib/python2.7/site-packages
        chardet>=3.0.2: 3.0.4 from /usr/lib/python2.7/site-packages
        idna>=2.5: 2.7 from /usr/lib/python2.7/site-packages
        urllib3>=1.21.1: 1.24.1 from /usr/lib/python2.7/site-packages
      setuptools: 40.5.0 from /usr/lib/python2.7/site-packages
      tornado>=4.4: 5.1.1 from /usr/lib/python2.7/site-packages
        futures: 3.2.0 from /usr/lib/python2.7/site-packages
        singledispatch: 3.4.0.3 from /usr/lib/python2.7/site-packages
          six: 1.11.0 from /usr/lib/python2.7/site-packages
        backports_abc>=0.4: 0.5 from /usr/lib/python2.7/site-packages
    Pykka>=1.1: 1.2.1 from /usr/lib/python2.7/site-packages
    uritools>=1.0: 1.0.1 from /usr/lib/python2.7/site-packages
      ipaddress>=1.0.6: 1.0.22 from /usr/lib/python2.7/site-packages
      ipaddress>=1.0.6: 1.0.22 from /usr/lib/python2.7/site-packages
  ConfigObj>=5.0.6: 5.0.6 from /usr/lib/python2.7/site-packages
  raven>=6.1.0: 6.9.0 from /usr/lib/python2.7/site-packages
    contextlib2: 0.5.5 from /usr/lib/python2.7/site-packages
Mopidy-Local-Images: 1.0.0 from /usr/lib/python2.7/site-packages
  setuptools: 40.5.0 from /usr/lib/python2.7/site-packages
  Mopidy>=1.1: 2.2.1 from /usr/lib/python2.7/site-packages
    Pykka>=1.1: 1.2.1 from /usr/lib/python2.7/site-packages
    requests>=2.0: 2.20.1 from /usr/lib/python2.7/site-packages
      chardet>=3.0.2: 3.0.4 from /usr/lib/python2.7/site-packages
      idna>=2.5: 2.7 from /usr/lib/python2.7/site-packages
      urllib3>=1.21.1: 1.24.1 from /usr/lib/python2.7/site-packages
    setuptools: 40.5.0 from /usr/lib/python2.7/site-packages
    tornado>=4.4: 5.1.1 from /usr/lib/python2.7/site-packages
      futures: 3.2.0 from /usr/lib/python2.7/site-packages
      singledispatch: 3.4.0.3 from /usr/lib/python2.7/site-packages
        six: 1.11.0 from /usr/lib/python2.7/site-packages
      backports_abc>=0.4: 0.5 from /usr/lib/python2.7/site-packages
  Pykka>=1.1: 1.2.1 from /usr/lib/python2.7/site-packages
  uritools>=1.0: 1.0.1 from /usr/lib/python2.7/site-packages
    ipaddress>=1.0.6: 1.0.22 from /usr/lib/python2.7/site-packages
    ipaddress>=1.0.6: 1.0.22 from /usr/lib/python2.7/site-packages
Mopidy-Spotify-Tunigo: 1.0.0 from /usr/lib/python2.7/site-packages
  Mopidy>=0.19.0: 2.2.1 from /usr/lib/python2.7/site-packages
    Pykka>=1.1: 1.2.1 from /usr/lib/python2.7/site-packages
    requests>=2.0: 2.20.1 from /usr/lib/python2.7/site-packages
      chardet>=3.0.2: 3.0.4 from /usr/lib/python2.7/site-packages
      idna>=2.5: 2.7 from /usr/lib/python2.7/site-packages
      urllib3>=1.21.1: 1.24.1 from /usr/lib/python2.7/site-packages
    setuptools: 40.5.0 from /usr/lib/python2.7/site-packages
    tornado>=4.4: 5.1.1 from /usr/lib/python2.7/site-packages
      futures: 3.2.0 from /usr/lib/python2.7/site-packages
      singledispatch: 3.4.0.3 from /usr/lib/python2.7/site-packages
        six: 1.11.0 from /usr/lib/python2.7/site-packages
      backports_abc>=0.4: 0.5 from /usr/lib/python2.7/site-packages
  Mopidy-Spotify>=1.2.0: 3.1.0 from /usr/lib/python2.7/site-packages
    Mopidy>=2.0: 2.2.1 from /usr/lib/python2.7/site-packages
      Pykka>=1.1: 1.2.1 from /usr/lib/python2.7/site-packages
      requests>=2.0: 2.20.1 from /usr/lib/python2.7/site-packages
        chardet>=3.0.2: 3.0.4 from /usr/lib/python2.7/site-packages
        idna>=2.5: 2.7 from /usr/lib/python2.7/site-packages
        urllib3>=1.21.1: 1.24.1 from /usr/lib/python2.7/site-packages
      setuptools: 40.5.0 from /usr/lib/python2.7/site-packages
      tornado>=4.4: 5.1.1 from /usr/lib/python2.7/site-packages
        futures: 3.2.0 from /usr/lib/python2.7/site-packages
        singledispatch: 3.4.0.3 from /usr/lib/python2.7/site-packages
          six: 1.11.0 from /usr/lib/python2.7/site-packages
        backports_abc>=0.4: 0.5 from /usr/lib/python2.7/site-packages
    Pykka>=1.1: 1.2.1 from /usr/lib/python2.7/site-packages
    pyspotify>=2.0.5: 2.0.5 from /usr/lib/python2.7/site-packages
      cffi>=1.0.0: 1.11.5 from /usr/lib/python2.7/site-packages
        pycparser: 2.19 from /usr/lib/python2.7/site-packages
    requests>=2.0: 2.20.1 from /usr/lib/python2.7/site-packages
      chardet>=3.0.2: 3.0.4 from /usr/lib/python2.7/site-packages
      idna>=2.5: 2.7 from /usr/lib/python2.7/site-packages
      urllib3>=1.21.1: 1.24.1 from /usr/lib/python2.7/site-packages
    setuptools: 40.5.0 from /usr/lib/python2.7/site-packages
  Pykka>=1.1: 1.2.1 from /usr/lib/python2.7/site-packages
  setuptools: 40.5.0 from /usr/lib/python2.7/site-packages
  tunigo>=1.0.0: 1.0.0 from /usr/lib/python2.7/site-packages
    requests>=2.0.0: 2.20.1 from /usr/lib/python2.7/site-packages
      chardet>=3.0.2: 3.0.4 from /usr/lib/python2.7/site-packages
      idna>=2.5: 2.7 from /usr/lib/python2.7/site-packages
      urllib3>=1.21.1: 1.24.1 from /usr/lib/python2.7/site-packages
    setuptools: 40.5.0 from /usr/lib/python2.7/site-packages
Mopidy-SoundCloud: 2.1.0 from /usr/lib/python2.7/site-packages
  setuptools: 40.5.0 from /usr/lib/python2.7/site-packages
  Mopidy>=1.0: 2.2.1 from /usr/lib/python2.7/site-packages
    Pykka>=1.1: 1.2.1 from /usr/lib/python2.7/site-packages
    requests>=2.0: 2.20.1 from /usr/lib/python2.7/site-packages
      chardet>=3.0.2: 3.0.4 from /usr/lib/python2.7/site-packages
      idna>=2.5: 2.7 from /usr/lib/python2.7/site-packages
      urllib3>=1.21.1: 1.24.1 from /usr/lib/python2.7/site-packages
    setuptools: 40.5.0 from /usr/lib/python2.7/site-packages
    tornado>=4.4: 5.1.1 from /usr/lib/python2.7/site-packages
      futures: 3.2.0 from /usr/lib/python2.7/site-packages
      singledispatch: 3.4.0.3 from /usr/lib/python2.7/site-packages
        six: 1.11.0 from /usr/lib/python2.7/site-packages
      backports_abc>=0.4: 0.5 from /usr/lib/python2.7/site-packages
  Pykka>=1.1: 1.2.1 from /usr/lib/python2.7/site-packages
  requests>=2.0.0: 2.20.1 from /usr/lib/python2.7/site-packages
    chardet>=3.0.2: 3.0.4 from /usr/lib/python2.7/site-packages
    idna>=2.5: 2.7 from /usr/lib/python2.7/site-packages
    urllib3>=1.21.1: 1.24.1 from /usr/lib/python2.7/site-packages
Mopidy-Spotify: 3.1.0 from /usr/lib/python2.7/site-packages
  Mopidy>=2.0: 2.2.1 from /usr/lib/python2.7/site-packages
    Pykka>=1.1: 1.2.1 from /usr/lib/python2.7/site-packages
    requests>=2.0: 2.20.1 from /usr/lib/python2.7/site-packages
      chardet>=3.0.2: 3.0.4 from /usr/lib/python2.7/site-packages
      idna>=2.5: 2.7 from /usr/lib/python2.7/site-packages
      urllib3>=1.21.1: 1.24.1 from /usr/lib/python2.7/site-packages
    setuptools: 40.5.0 from /usr/lib/python2.7/site-packages
    tornado>=4.4: 5.1.1 from /usr/lib/python2.7/site-packages
      futures: 3.2.0 from /usr/lib/python2.7/site-packages
      singledispatch: 3.4.0.3 from /usr/lib/python2.7/site-packages
        six: 1.11.0 from /usr/lib/python2.7/site-packages
      backports_abc>=0.4: 0.5 from /usr/lib/python2.7/site-packages
  Pykka>=1.1: 1.2.1 from /usr/lib/python2.7/site-packages
  pyspotify>=2.0.5: 2.0.5 from /usr/lib/python2.7/site-packages
    cffi>=1.0.0: 1.11.5 from /usr/lib/python2.7/site-packages
      pycparser: 2.19 from /usr/lib/python2.7/site-packages
  requests>=2.0: 2.20.1 from /usr/lib/python2.7/site-packages
    chardet>=3.0.2: 3.0.4 from /usr/lib/python2.7/site-packages
    idna>=2.5: 2.7 from /usr/lib/python2.7/site-packages
    urllib3>=1.21.1: 1.24.1 from /usr/lib/python2.7/site-packages
  setuptools: 40.5.0 from /usr/lib/python2.7/site-packages
GStreamer: 1.14.4.0 from /usr/lib/python2.7/site-packages/gi
  Detailed information: 
    Python wrapper: python-gi 3.30.2
    Relevant elements:
      Found:
        uridecodebin
        souphttpsrc
        appsrc
        alsasink
        osssink
        oss4sink
        pulsesink
        id3demux
        id3v2mux
        lamemp3enc
        mpegaudioparse
        mpg123audiodec
        vorbisdec
        vorbisenc
        vorbisparse
        oggdemux
        oggmux
        oggparse
        flacdec
        flacparse
        shout2send
      Not found:
        flump3dec
        mad

I'm running into an issue trying to get udpsink working with Mopidy and all remote ncmpcpp clients. Mopidy is setup with MacVLAN in a docker environment. I can successfully see the port on the container.

nc -vz -u mopidy.lan 5555
found 0 associations
found 1 connections:
     1: flags=82<CONNECTED,PREFERRED>
    outif (null)
    src x.x.x.x port 50630
    dst x.x.x.x port 5555
    rank info not available

Connection to mopidy.lan port 5555 [udp/personal-agent] succeeded!

Currently Mopidy is using the following bit of config:

[audio]
mixer = software
mixer_volume = 100
output = tee name=t ! queue ! lamemp3enc ! shout2send async=false mount=mopidy ip="mopidy.lan" port=8092 username="some username" password="some password" t. ! queue ! udpsink host=0.0.0.0 port=5555

Any thoughts to how I can use netcat to successfully get the raw data into mpd.fifo? Using the following doesn't work, I think because I'm using netcat in the wrong way. Some research hasn't provided any answers, so hopefully someone can point me in the right direction.

#!/bin/bash

mkfifo /tmp/mpd.fifo
while :; do yes $'\n' | nc -lu mopidy 5555 > /tmp/mpd.fifo; done

I get the following error:

nc: Can't assign requested address

Any thoughts on how I can get this to work?

Cheers!

FYI I added support for gstreamer's udpsink to ncmpcpp so now it can be used directly without fifo hacks.

See https://github.com/ncmpcpp/ncmpcpp/commit/fb886f687014e22b2fe1477da855be5201063ea8 for more details.

Below a teaser. This is mopidy + mopidy-spotify + mopidy-mpd + ncmpcpp.

https://user-images.githubusercontent.com/387658/102549720-e41bf980-40bc-11eb-8127-70889011e52f.mp4

@arybczak Awesome! You should post this to https://discourse.mopidy.com/c/show-and-tell/9

@adamcik Doesn't this mean that this nearly 7-yo issue is "deprecated" in favour of a better resolution to the real-world issue that sparked it (supporting visualisations in ncmpcpp)?

@pspeder, what exactly do you mean? Is there a problem with using https://github.com/mopidy/mopidy/issues/775#issuecomment-747725806 ?

@kingosticks Was simply suggesting that this issue be closed 🙂

I do agree with that.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

handsomegui picture handsomegui  ·  12Comments

ecoCuyo picture ecoCuyo  ·  3Comments

mczerski picture mczerski  ·  9Comments

pnijhara picture pnijhara  ·  5Comments

szuniverse picture szuniverse  ·  13Comments