Aiohttp: Websocket客户端挂起/阻止?

创建于 2018-09-26  ·  12评论  ·  资料来源: aio-libs/aiohttp

长话短说

尽管有数据从websocket流出,但ws.receive()似乎正在挂起。

预期行为

ws.receive应该输出并报告某些内容。

实际行为

它只是挂在/阻止在websocket上。

重现步骤

下面的代码是调用代码。

    async def _ws_subscribe(self,
                            session: ClientSession,
                            url: str
                            ) -> Dict:
        """ Fetch data from web socket asynchronously.

        This helper method fetches data from a web socket asynchronously. It is used to
        fetch subscribe to websockets of the SkyQ box.

        Args:
            session (aiohttp.ClientSession): Session to use when fetching the data.
            url (str): WebSocket URL to fetch.

        Returns:
            dict: The body of data returned.

        """
        LOGGER.debug(f"Inside _ws_subscribe()...")

        async with session.ws_connect(url, autoclose=False) as ws:
            while True:
                LOGGER.debug(f"Inside _ws_subscribe() infinite loop pre ws.receive.")
                payload = await ws.receive()
                LOGGER.debug(f"READ SOMETHING! _ws_subscribe() infinite loop post ws.receive")
                LOGGER.debug(f'type = {payload.type}')
                LOGGER.debug(f'payload data = {payload.data}')
                LOGGER.debug(f'payload exception = {ws.exception()}')
                if payload.type == aiohttp.WSMsgType.TEXT:
                    LOGGER.debug('Web-socket data received.')
                    asyncio.create_task(self._handle(payload))
                    # asyncio.ensure_future(self._handle(payload))
                elif payload.type == aiohttp.WSMsgType.BINARY:
                    LOGGER.debug('Web-socket binary data received.')
                elif payload.type == aiohttp.WSMsgType.PING:
                    LOGGER.debug('Web-socket ping received')
                    ws.pong()
                elif payload.type == aiohttp.WSMsgType.PONG:
                    LOGGER.debug('Web-socket pong received')
                else:
                    if payload.type == aiohttp.WSMsgType.CLOSE:
                        await ws.close()
                    elif payload.type == aiohttp.WSMsgType.ERROR:
                        LOGGER.info(f'Error during receive {ws.exception()}')
                    elif payload.type == aiohttp.WSMsgType.CLOSED:
                        pass

                    break

结果日志:

(pyskyq-4vSEKDfZ) ✔ [brad<strong i="6">@bradmac</strong>:~/Code/pyskyq] [16-create-status-end-point-property|✚ 5…2] $ pyskyq -vv green
[2018-09-26 09:12:14] DEBUG:pyskyq.cli:Starting SkyQ...
[2018-09-26 09:12:14] DEBUG:pyskyq.skyremote:Initialised SkyRemote object with host=skyq, port=49160
[2018-09-26 09:12:14] DEBUG:pyskyq.status:Initialised Status object object with host=skyq, port=9006
[2018-09-26 09:12:14] DEBUG:asyncio:Using selector: KqueueSelector
[2018-09-26 09:12:14] DEBUG:pyskyq.status:Asyncio event loop thread running...
[2018-09-26 09:12:14] DEBUG:pyskyq.skyq:Initialised SkyQ object with host=skyq.
[2018-09-26 09:12:14] INFO:pyskyq.cli:Script ends here
[2018-09-26 09:12:14] DEBUG:pyskyq.status:Setting up web socket listener on ws://skyq:9006/as/system/status.
[2018-09-26 09:12:14] DEBUG:pyskyq.status:Inside _ws_subscribe()...
[2018-09-26 09:12:14] DEBUG:asyncio:Get address info skyq:9006, type=<SocketKind.SOCK_STREAM: 1>
[2018-09-26 09:12:14] DEBUG:asyncio:Getting address info skyq:9006, type=<SocketKind.SOCK_STREAM: 1> took 15.352ms: [(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('10.0.1.6', 9006))]
[2018-09-26 09:12:14] DEBUG:asyncio:poll 60478.847 ms took 14.299 ms: 1 events
[2018-09-26 09:12:14] DEBUG:asyncio:connect <socket.socket fd=8, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('0.0.0.0', 0)> to ('10.0.1.6', 9006)
[2018-09-26 09:12:14] DEBUG:asyncio:poll 60461.657 ms took 21.045 ms: 1 events
[2018-09-26 09:12:14] DEBUG:asyncio:<socket.socket fd=8, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.1.30', 50141), raddr=('10.0.1.6', 9006)> connected to 10.0.1.6:9006: (<_SelectorSocketTransport fd=8 read=polling write=<idle, bufsize=0>>, <aiohttp.client_proto.ResponseHandler object at 0x1033c8ac8>)
[2018-09-26 09:12:14] DEBUG:asyncio:poll 60429.970 ms took 7.192 ms: 1 events
[2018-09-26 09:12:14] DEBUG:pyskyq.status:Inside _ws_subscribe() infinite loop pre ws.receive.

但是,套接字是打开的,并且可以通过netcat看到数据。

(pyskyq-4vSEKDfZ) ✔ [brad<strong i="6">@bradmac</strong>:~/Code/pyskyq] [16-create-status-end-point-property|✚ 5…2] $ nc skyq 9006
GET /as/system/status HTTP/1.1
Host: skyq:9006
Accept: */*
Connection: Upgrade
Upgrade: websocket

�~�{
   "camessage" : {
      "reason" : "no message",
      "state" : "unavailable"
   },
   "drmstatus" : {
      "state" : "available"
   },
   "entitlements" : [
      "ANALYTICS",
      "BIGBASIC",
      "ETHAN_APP_1",
      "HD",
      "PDL",
      "SKY_DRM_CE",
      "SKY_DRM_MR",
      "SKY_IPPV",
      "ULTRA+",
      "SKY+",
      "GATEWAYENABLER",
      "SIDELOAD"
   ],
   "epginfobits" : {
      "epginfobits" : "0xFDE5FFC0",
      "mask" : "0x5FFA003F",
      "state" : "available"
   },
   "gatewayservices" : {
      "state" : "available"
   },
   "hdmi" : {
      "2160p10bitCapable" : false,
      "authenticatedHDCP" : "1.x",
      "sinkHDCP" : "1.x",
      "sinkHLG" : false,
      "sinkUHD" : false,
      "state" : "available",
      "uhdConfigured" : false
   },
   "network" : {
      "state" : "available"
   },
   "nssplayback" : {
      "state" : "available"
   },
   "pvr" : {
      "state" : "available"
   },
   "schedule" : {
      "lastdate" : "20181003",
      "state" : "available"
   },
   "servicelist" : {
      "state" : "available"
   },
   "smartcard" : {
      "active" : true,
      "bouquet" : "4101",
      "countryCode" : "GBR",
      "currency" : "GBP",
      "cwe" : true,
      "householdid" : "10947783",
      "paired" : true,
      "state" : "available",
      "subbouquet" : "1",
      "transactionlimit" : 65535,
      "viewingCardNumber" : "725 325 260"
   },
   "swupdate" : {
      "reason" : "IDLE",
      "state" : "unavailable"
   },
   "systemupdates" : {
      "entitlements" : 2,
      "install" : 1,
      "servicegenres" : 1,
      "smartcard" : 1
   },
   "updatetask" : {
      "reason" : "no update",
      "state" : "unavailable"
   }
}
^C(pyskyq-4vSEKDfZ) ✔ [brad<strong i="7">@bradmac</strong>:~/Code/pyskyq] [16-create-status-end-point-property|✚ 5…2] $

您的环境

(pyskyq-4vSEKDfZ) ✔ [brad<strong i="6">@bradmac</strong>:~/Code/pyskyq] [16-create-status-end-point-property|✚ 5…2] $ uname -a
Darwin bradmac 17.7.0 Darwin Kernel Version 17.7.0: Thu Jun 21 22:53:14 PDT 2018; root:xnu-4570.71.2~1/RELEASE_X86_64 x86_64
(pyskyq-4vSEKDfZ) ✔ [brad<strong i="7">@bradmac</strong>:~/Code/pyskyq] [16-create-status-end-point-property|✚ 5…2] $ pip freeze
aiohttp==3.4.4
appnope==0.1.0
arrow==0.12.1
async-timeout==3.0.0
async-upnp-client==0.12.4
attrs==18.2.0
backcall==0.1.0
binaryornot==0.4.4
bleach==2.1.3
certifi==2018.4.16
chardet==3.0.4
click==6.7
cookiecutter==1.6.0
cycler==0.10.0
decorator==4.3.0
Django==2.1.1
entrypoints==0.2.3
ez-setup==0.9
future==0.16.0
html5lib==1.0.1
idna==2.7
idna-ssl==1.1.0
ipykernel==4.8.2
ipython==6.4.0
ipython-genutils==0.2.0
ipywidgets==7.2.1
jedi==0.12.0
Jinja2==2.10
jinja2-time==0.2.0
jsonschema==2.6.0
jupyter==1.0.0
jupyter-client==5.2.3
jupyter-console==5.2.0
jupyter-core==4.4.0
kaggle==1.4.2
kiwisolver==1.0.1
MarkupSafe==1.0
matplotlib==2.2.2
mistune==0.8.3
multidict==4.4.2
nbconvert==5.3.1
nbformat==4.4.0
notebook==5.5.0
numpy==1.14.4
pandocfilters==1.4.2
parso==0.2.1
pexpect==4.6.0
pickleshare==0.7.4
pipenv==2018.7.1
poyo==0.4.1
prompt-toolkit==1.0.15
ptyprocess==0.6.0
Pygments==2.2.0
pyparsing==2.2.0
PyScaffold==3.0.3
python-dateutil==2.7.3
python-didl-lite==1.1.0
pytz==2018.4
pyzmq==17.0.0
qtconsole==4.3.1
requests==2.19.1
Send2Trash==1.5.0
simplegeneric==0.8.1
six==1.11.0
terminado==0.8.1
testpath==0.3.1
tornado==5.0.2
tqdm==4.24.0
traitlets==4.3.2
urllib3==1.22
virtualenv==16.0.0
virtualenv-clone==0.3.0
voluptuous==0.11.5
wcwidth==0.1.7
webencodings==0.5.1
whichcraft==0.4.1
widgetsnbextension==3.2.1
yarl==1.2.6
(pyskyq-4vSEKDfZ) ✔ [brad<strong i="8">@bradmac</strong>:~/Code/pyskyq] [16-create-status-end-point-property|✚ 5…2] $

更新...

Python版本是3.7.0

所有12条评论

GitMate.io认为可能相关的问题是https://github.com/aio-libs/aiohttp/issues/2200 (后台客户端websockets挂在SIGINT上), https://github.com/aio-libs/aiohttp/issues/ 1002 (Websocket响应.close()可以无限期挂起), https: //github.com/aio-libs/aiohttp/issues/376 (ProactorEventLoop hangs), https://github.com/aio-libs/aiohttp/issues / 3027 (ws_connect挂起)和https://github.com/aio-libs/aiohttp/issues/265 (实现客户端websocket)。

看起来client_ws.py#L204没有返回...任何想法?

该服务器是SkyQ机顶盒,因此可能不是完全符合RFC的WS实现。 例如,它不执​​行ping / poings操作。有效负载中有一个奇怪的�~�

Websocket是二进制协议�~�对消息类型和长度进行编码。
抱歉,如果您不参与解码并将其与实际有效负载大小进行比较,我将无法为您提供帮助。

�~�是以二进制形式的001111110111111000111111 ,即:以十六进制表示的3F7E3F

这些字节看起来像是端到端的...
00111111
01111110
00111111

我不是RFC方面的专家,但是在类型和长度方面这是什么意思?

您认为问题出在那吗? 也就是说,例如,它不同意实际的有效载荷长度,并且等待额外的帧吗?

有趣的是,这个用go编写的

```
ws ws:// skyq:9006 / as / system / status
<{
“摄像头”:{
“ reason”:“没有消息”,
“ state”:“不可用”
},
“ drmstatus”:{
“ state”:“可用”
},
“权利”:[
“分析”,
“ BIGBASIC”,
“ ETHAN_APP_1”,
“高清”,
“ PDL”,
“ SKY_DRM_CE”,
“ SKY_DRM_MR”,
“ SKY_IPPV”,
“ ULTRA +”,
“ SKY +”,
“ GATEWAYENABLER”,
“ SIDELOAD”
],
“ epginfobits”:{
“ epginfobits”:“ 0xFDE5FFC0”,
“ mask”:“ 0x5FFA003F”,
“ state”:“可用”
},
“ gatewayservices”:{
“ state”:“可用”
},
“ hdmi”:{
“ 2160p10bitCapable”:false,
“ authenticatedHDCP”:“ NONE”,
“ reason”:“ HDMI输出端口已禁用”,
“ sinkHDCP”:“ NONE”,
“ sinkHLG”:否,
“ sinkUHD”:否,
“ state”:“不可用”,
“ uhdConfigured”:false
},
“网络” : {
“ state”:“可用”
},
“ nssplayback”:{
“ state”:“可用”
},
“ pvr”:{
“ state”:“可用”
},
“日程” : {
“ lastdate”:“ 20181003”,
“ state”:“可用”
},
“服务列表”:{
“ state”:“可用”
},
“智能卡” : {
“ active”:是的,
“花束”:“ 4101”,
“ countryCode”:“ GBR”,
“ currency”:“ GBP”,
“ cwe”:是的,
“ householdid”:“ 10947783”,
“成对的”:是的,
“ state”:“ available”,
“ subbouquet”:“ 1”,
“ transactionlimit”:65535,
“ viewingCardNumber”:“ 725 325 260”
},
“ swupdate”:{
“ reason”:“ IDLE”,
“ state”:“不可用”
},
“系统升级” : {
“权利”:2
“安装”:1,
“服务类型”:1
“智能卡”:1
},
“ updatetask”:{
“ reason”:“没有更新”,
“ state”:“不可用”
}
}

^ C
打断
✔[ brad @ bradmac :〜] $```

该协议说:WS消息中的第一个3F字节是禁止的(保留以备将来使用)。
您确定您没有错过任何东西吗?

我假设您指的是基本成帧协议中的2d,3rd和4th位。 在这种情况下,也许我确实弄错了。 可能是unicode / ASCII问题。 我再看一看。

我已经尝试了其他一些在线转换器...现在,我得到了:
十六进制= ef bf bd 7e ef bf bd
二进制= 11101111 10111111 10111101 01111110 11101111 10111111 10111101

如果根据RFC 6455

5.2.  Base Framing Protocol

   This wire format for the data transfer part is described by the ABNF
   [RFC5234] given in detail in this section.  (Note that, unlike in
   other sections of this document, the ABNF in this section is
   operating on groups of bits.  The length of each group of bits is
   indicated in a comment.  When encoded on the wire, the most
   significant bit is the leftmost in the ABNF).  A high-level overview
   of the framing is given in the following figure.  In a case of
   conflict between the figure below and the ABNF specified later in
   this section, the figure is authoritative.

      0                   1                   2                   3
      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
     +-+-+-+-+-------+-+-------------+-------------------------------+
     |F|R|R|R| opcode|M| Payload len |    Extended payload length    |
     |I|S|S|S|  (4)  |A|     (7)     |             (16/64)           |
     |N|V|V|V|       |S|             |   (if payload len==126/127)   |
     | |1|2|3|       |K|             |                               |
     +-+-+-+-+-------+-+-------------+ - - - - - - - - - - - - - - - +
     |     Extended payload length continued, if payload len == 127  |
     + - - - - - - - - - - - - - - - +-------------------------------+
     |                               |Masking-key, if MASK set to 1  |
     +-------------------------------+-------------------------------+
     | Masking-key (continued)       |          Payload Data         |
     +-------------------------------- - - - - - - - - - - - - - - - +
     :                     Payload Data continued ...                :
     + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +
     |                     Payload Data continued ...                |
     +---------------------------------------------------------------+

   FIN:  1 bit

      Indicates that this is the final fragment in a message.  The first
      fragment MAY also be the final fragment.

   RSV1, RSV2, RSV3:  1 bit each

      MUST be 0 unless an extension is negotiated that defines meanings
      for non-zero values.  If a nonzero value is received and none of
      the negotiated extensions defines the meaning of such a nonzero
      value, the receiving endpoint MUST _Fail the WebSocket
      Connection_.

然后,就像您建议的那样,似乎RSV1RSV2被设置为1,除非正在协商某些扩展? 我不知道。

JSON文档中的字符数为1703

RFC关于有效负载长度说明了这一点:

Payload length:  7 bits, 7+16 bits, or 7+64 bits

      The length of the "Payload data", in bytes: if 0-125, that is the
      payload length.  If 126, the following 2 bytes interpreted as a
      16-bit unsigned integer are the payload length.  If 127, the
      following 8 bytes interpreted as a 64-bit unsigned integer (the
      most significant bit MUST be 0) are the payload length.  Multibyte
      length quantities are expressed in network byte order.  Note that
      in all cases, the minimal number of bytes MUST be used to encode
      the length, for example, the length of a 124-byte-long string
      can't be encoded as the sequence 126, 0, 124.  The payload length
      is the length of the "Extension data" + the length of the
      "Application data".  The length of the "Extension data" may be
      zero, in which case the payload length is the length of the
      "Application data".

那么,考虑到所有这些,我们下一步要去哪里? 如果服务器死于大脑并且不符合RFC,那么我需要解决它,因为其他客户端显然可以做到...您能提供任何建议吗?

看起来仍然可疑。
唯一已知的扩展是利用RSV1的WebSocket压缩。 我不知道什么扩展使用RSV2。
此外,操作码FF也是保留的,不应使用。

除非您不知道服务器发送的内容,否则无法“修复”客户端。

@asvetlov我尝试了websockets库,它可以工作...所以我将继续使用它,因为我不能花更多的时间与您一起调试该库...

aoihttp -不起作用

import asyncio
import logging
import sys

import aiohttp

LOGGER = logging.getLogger(__name__)
logformat = "[%(asctime)s] %(levelname)s:%(name)s:%(message)s"
logging.basicConfig(level=logging.DEBUG, stream=sys.stdout,
                    format=logformat, datefmt="%Y-%m-%d %H:%M:%S")


async def main():
    async with aiohttp.ClientSession() as session:
        async with session.ws_connect('http://skyq:9006/as/system/status') as ws:
            payload = await ws.receive()
            print(payload.text)

loop = asyncio.get_event_loop()
loop.run_until_complete(main())

websockets -作品

import asyncio
import websockets
import logging
import sys

LOGGER = logging.getLogger(__name__)
logformat = "[%(asctime)s] %(levelname)s:%(name)s:%(message)s"
logging.basicConfig(level=logging.DEBUG, stream=sys.stdout,
                    format=logformat, datefmt="%Y-%m-%d %H:%M:%S")


async def hello():
    async with websockets.connect(
            'ws://skyq:9006/as/system/status') as websocket:

        payload = await websocket.recv()
        print(f"{payload}")

asyncio.get_event_loop().run_until_complete(hello())

如果愿意,可以免费关闭这张票,但是您可能想要使其保持打开状态,因为它似乎是一个错误……或者至少是一种不遵守Postel法则的情况。

感谢您为解决此问题所提供的帮助。

看起来仍然可疑。
唯一已知的扩展是利用RSV1的WebSocket压缩。 我不知道什么扩展使用RSV2。
此外,操作码FF也是保留的,不应使用。

@asvetlov

除非您不知道服务器发送的内容,否则无法“修复”客户端。

我想你的意思是“除非,知道什么是服务器发送,你不能修复的客户端。”

在这种情况下,我完全不同意这一点。

您熟悉稳健性原则吗?

“对发送的内容要保守,对接收的内容要宽松。”

不管必须,应该和不应该,在RFC中,我认为您都在解释并期望实体完全遵守RFC。……显然,它们并没有,并且许多不同类型的互联网服务器和客户端都拥有RFC。可怕的混蛋协议的一半实现。

一个好的库/协议实现必须在可能的范围内原谅这些。 参见RFC1122 。 完全有可能编写协议客户端以免除(某些)协议违规,而无需访问通信中的其他参与者。

我觉得这里不是这种情况,您似乎持强硬观点,即如果主机不100%兼容,那么它就太糟糕了,我们将不予讨论。 太可惜了就像是在说:“我只说女王的英语,如果您用某种奇怪的英语方言来跟我说话,我就不会和您说话。”

显然,如果整个图书馆都想促进交流,那么这不是一个好方法。

因此,我的建议是考虑可以在哪里放宽协议的严格要求(也许通过方法kwargs,打开“松弛”处理),看看即使对方表现不佳,也可以使事情畅通无阻。 。

除了这种批评之外,我还要感谢您和aio-libs的贡献者为社区所做的所有工作。 🥇

@asvetlov只是对此的更新...我发现我的(非兼容)服务器需要一个大写的Upgrade:标头,然后它才能工作。 只是想我提到过,以防你们正在考虑标题案例标题的问题……这是来自实施不良的服务器的协议违规的真实案例,导致websocket客户端无法正常工作。

此页面是否有帮助?
0 / 5 - 0 等级