Fabric: python3下并行执行

创建于 2018-12-02  ·  14评论  ·  资料来源: fabric/fabric

我找不到任何有关较新结构版本的并行执行的文档。 它与python 2版本相同,还是正在进行中?

谢谢!

最有用的评论

@akiuni有这方面的公关吗? 第1912章(评论)

你好,我不这么认为.. 老实说,我知道如何检查,我不知道如何继续提交这样的修改......

所有14条评论

是的,但是,线程组不支持配置,特别是:

config = Config(overrides={'sudo':{'password': 'changeme'}})

c = ThreadingGroup("[email protected]", "[email protected]", connect_kwargs={"password": "changeme"}, config=config)

c.run("whoami")

c.sudo("whoami") #Throws AttributeError: 'ThreadingGroup' object has no attribute 'sudo'

ThreadingGroup 只允许运行命令。
https://github.com/fabric/fabric/issues/1899#issue -377714688

您可以重写并添加命令支持。

您如何看待以下在 SerialGroup 和 ThreadingGroup 中启用 sudo 的补丁?
(在织物 2.4 上应用和测试)

--- fabric-orig/group.py    2018-10-30 14:35:22.000000000 +0100
+++ fabric-patched/group.py 2019-02-18 10:16:29.186000000 +0100
@@ -183,12 +183,28 @@
             raise GroupException(results)
         return results

+    def sudo(self, *args, **kwargs):
+        results = GroupResult()
+        excepted = False
+        for cxn in self:
+            try:
+                results[cxn] = cxn.sudo(*args, **kwargs)
+            except Exception as e:
+                results[cxn] = e
+                excepted = True
+        if excepted:
+            raise GroupException(results)
+        return results

 def thread_worker(cxn, queue, args, kwargs):
     result = cxn.run(*args, **kwargs)
     # TODO: namedtuple or attrs object?
     queue.put((cxn, result))

+def thread_worker_sudo(cxn, queue, args, kwargs):
+    result = cxn.sudo(*args, **kwargs)
+    # TODO: namedtuple or attrs object?
+    queue.put((cxn, result))

 class ThreadingGroup(Group):
     """
@@ -208,6 +224,49 @@
             )
             threads.append(thread)
         for thread in threads:
+            thread.start()
+        for thread in threads:
+            # TODO: configurable join timeout
+            # TODO: (in sudo's version) configurability around interactive
+            # prompting resulting in an exception instead, as in v1
+            thread.join()
+        # Get non-exception results from queue
+        while not queue.empty():
+            # TODO: io-sleep? shouldn't matter if all threads are now joined
+            cxn, result = queue.get(block=False)
+            # TODO: outstanding musings about how exactly aggregate results
+            # ought to ideally operate...heterogenous obj like this, multiple
+            # objs, ??
+            results[cxn] = result
+        # Get exceptions from the threads themselves.
+        # TODO: in a non-thread setup, this would differ, e.g.:
+        # - a queue if using multiprocessing
+        # - some other state-passing mechanism if using e.g. coroutines
+        # - ???
+        excepted = False
+        for thread in threads:
+            wrapper = thread.exception()
+            if wrapper is not None:
+                # Outer kwargs is Thread instantiation kwargs, inner is kwargs
+                # passed to thread target/body.
+                cxn = wrapper.kwargs["kwargs"]["cxn"]
+                results[cxn] = wrapper.value
+                excepted = True
+        if excepted:
+            raise GroupException(results)
+        return results
+
+    def sudo(self, *args, **kwargs):
+        results = GroupResult()
+        queue = Queue()
+        threads = []
+        for cxn in self:
+            my_kwargs = dict(cxn=cxn, queue=queue, args=args, kwargs=kwargs)
+            thread = ExceptionHandlingThread(
+                target=thread_worker_sudo, kwargs=my_kwargs
+            )
+            threads.append(thread)
+        for thread in threads:
             thread.start()
         for thread in threads:
             # TODO: configurable join timeout

老实说,那真是太棒了@akiuni

最好将 execute() 功能返回到 Fabric2 的 ThreadingGroup。

@akiuni有这方面的公关吗? https://github.com/fabric/fabric/issues/1912#issuecomment -464652728

@akiuni有这方面的公关吗? 第1912章(评论)

你好,我不这么认为.. 老实说,我知道如何检查,我不知道如何继续提交这样的修改......

我们需要尽快将其合并到 Fabric 中。

mm 查看 v 2.5.0 中的当前代码
https://github.com/fabric/fabric/blob/2.5.0/fabric/group.py#L127我可以看到这段代码不再是最新的,作者正在考虑实现这一点的最佳方法。 同样没有 PR,他们没有平台来讨论代码引入的更改。 我正在看这个,但我不是专业的 Python 编码器,但我会尝试将其作为 PR

公关在这里: https ://github.com/fabric/fabric/pull/2052

你好,我不习惯 git developpemen(PR 对我来说是一个陌生的词),如果我能帮忙,请随时告诉我

嘿@akiuni
PR 代表 Pull Request,这意味着,如果您想进行更改,您可以通过 pull request 将更改带入源代码,然后可以将其视为对话并进行审查,直到获得批准。 当它被批准后,PR 引入的更改可以追溯到我在这里所做的提交:#2052
一个聪明的技巧是将“.patch”添加到拉取请求(PR)中,如下所示: https ://patch-diff.githubusercontent.com/raw/fabric/fabric/pull/2052.patch 然后会给你一个差异的更改(正如您在之前的评论中提供的那样),但现在,好处是,如上所述的对话/审查/单点更改 - 感谢您成为第一个为此提供补丁的人,我一直在寻找在我创建 PR 时

Sudo 正在添加到组中! 见#1999

此页面是否有帮助?
0 / 5 - 0 等级

相关问题

supriyopaul picture supriyopaul  ·  4评论

acdha picture acdha  ·  4评论

jamesob picture jamesob  ·  3评论

bitprophet picture bitprophet  ·  4评论

amezin picture amezin  ·  5评论