Plumber: Timeout with a long request

Created on 25 Aug 2016 · 3Comments · Source: rstudio/plumber

Hi !

We are generating a long report in PDF that takes around 3 minutes and we expose it via Plumber.
The HTTP connection close around 1 minutes 40 seconds (100 seconds).

So we have the logical error message because plumber try to send data to a closed connection:
ERROR: [on_request_read] connection reset by peer

I think it can be related to #12 and by the way https://github.com/rstudio/httpuv/issues/49

There is a way to fix that in Plumber? May be by sending manually in Plumber something like that:

Keep-Alive: timeout=300
Connection: Keep-Alive

But in any case we want to avoid #12

Best regards,
Emmanuel

Source

emmanuelchevallier

Most helpful comment

Just in case anyone finds this issue relevant: forking works quite nicely and is easily done with the parallel package.

parallel <- function() {
    # Long running calculation here
    print("Sleeping...")
    Sys.sleep(10)
    print("Finished.")
    return(Sys.time())
}

#* @get /parallel
parallelTest <- function() {
    parallel::mcparallel(parallel())
    return(TRUE)
}

semuelle on 16 Nov 2016

👍4

All 3 comments

Unfortunately I'm not familiar enough with the internals of httpuv to be able to speak intelligently about why this might happen.

To take a step back, though, the usual pattern when doing these kinds of long-lived tasks in other systems is to immediately respond to the incoming request and then kick off the work asynchronously. The client gets some ID in response to the original request and then can poll for it to find out when it's complete.

Unfortunately, R is single threaded so this is a bit more complicated. But if you were sufficiently motivated you could look at forking the R process (see the parallel or multicore packages) and then generate the report in a fork of the process, leaving the parent process available to respond to API requests (like clients who are polling to see if their report is generated).

trestletech on 13 Sep 2016

Hi,

Here, we have optimzed our R code so it's not a problem actually. But the issue can come back in other cases. If I found a solution I will come back and publish it here.

emmanuelchevallier on 16 Sep 2016

Just in case anyone finds this issue relevant: forking works quite nicely and is easily done with the parallel package.

parallel <- function() {
    # Long running calculation here
    print("Sleeping...")
    Sys.sleep(10)
    print("Finished.")
    return(Sys.time())
}

#* @get /parallel
parallelTest <- function() {
    parallel::mcparallel(parallel())
    return(TRUE)
}

semuelle on 16 Nov 2016

👍4

Was this page helpful?

0 / 5 - 0 ratings