Faraday: Why is there no close method?

Sounds good. We'll gladly accept a tested patch for this.

I currently don't have time and I am not in the faraday source. I will return to it if this changes :)

Would be nice if you keep this open as a reminder.

Cool. It may actually be easier in Faraday 0.10/1.0 when we drop the current rack-inspired builder. The "callback builder" has a direct link to an adapter, making it easy to proxy the #close call to the adapter. Currently, you'd have to loop through the middleware looking for anything that responds to #close. Actually, that's not a horrible idea, gives middleware the chance to clean up any resources too.

Keep in mind that net-http-persistent _may_ look like it's not reusing connections, but it is. All instances of Net::HTTP::Persistent use the same connection pool internally.

I've been looking around for a way to close connections as well.
I understand the difficulty of providing a consistent interface as each library has a different method (shutdown for net-http-persistent, finish for net-http, close for excon) for closing connections after they are established.
However, it would be a big win to be able to call Faraday::Connection#close to actively close the connection for all supported adapters and middleware.

This won't happen in Faraday most likely. #454 explains the decision to start focusing on Hurley (essentially a rewrite). The problem is that your Faraday::Connection doesn't know what middleware is the actual connection. It's just a stack of procs. Hurley solves this by separating the connection adapter from the stack of callbacks. However, Hurley's current interface doesn't support the notion of a close operation either :) But due to its design, it's much easier to add it to Hurley, than Faraday. I'll try to look into this soon, and ping you on the PR if you're interested.

Thanks @technoweenie that would be great and that explanation makes sense.

From my understanding, Hurley is dead -- any thoughts on re-opening this request if that's true? We'd love to have a close method, since we're running into a lot of "Net::HTTP::Persistent::Error: too many connection resets (due to Connection reset by peer - Errno::ECONNRESET)" errors and would like to try force-closing the connection afterwards, but this issue makes that challenging.

Hi all,

I would be happy to bring this back to life for a future Faraday release.
I personally like the idea of having a method in all Middlewares that could be called when the connection is closed (some sort of callback, if you want).

Happy to discuss about it here if anyone have some time to work on it 👍

The problem is that your Faraday::Connection doesn't know what middleware is the actual connection.

We do this in async-http, and we just pass the #close method thorough all the middlewares so they can do their own work, e.g. releasing caches, whatever.

Please make this part of the 1.0 release. Even if it's not implemented in all middleware.

You should also document thread-safety of middleware and the expectations of user code, i.e. are adapters expected to be thread safe, invoked from different threads, etc. This is all part of the life cycle management.

e.g. in async-http, we don't share persistent pools between threads. In CRuby we spawn separate processes within Falcon which mostly avoids these issues, but in JRuby we only have different threads.

There's an effort in #1024 to be smarter about when to create connection objects, whether to pool them, etc. This could to a case where a Faraday::Connection#close could close the correct http connection object.

I really like this idea too:

and we just pass the #close method thorough all the middlewares so they can do their own work, e.g. releasing caches, whatever.

I'd really like to get all this into 1.0, but I need help in the way of code contributions and user feedback.

As for the thread safety, there are no guarantees unfortunately. Ideally, I'd like to get to a place where Faraday::Connection, including middleware and adapters, are all threadsafe.

If you can’t guarantee thread safety, why don’t you add a middleware at the very top of the stack with a re-entrant mutex to ensure that only one thread can use the rest of the stack at a time.

Regarding close, why don’t you just add this in the base class for middleware. The default implementation can just call close on the next middleware.

What do you think?

Can you change milestone to 1.0?

and we just pass the #close method thorough all the middlewares so they can do their own work, e.g. releasing caches, whatever.

I have an idea on how to implement this, however it won't be bullet-proof due to the freedom we give to push anything in the middleware stack as far as it answers call. It should still work fine in 90% of cases though.

Regarding the thread safety, although we'd love to get there, I guess our biggest issue is the lack of someone in the core team knowledgeable enough on the matter to help us achieving that.

@iMacTia what about my PR?

There is a #close method now, all thanks to #1069. Implementing it properly in middleware (such as the net-http-persistent adapter) should be left for separate pull requests.

There is an effort in https://github.com/lostisland/faraday/issues/1024 to refactor the adapters to allow for cached client instances (such as in the net-http-persistent adapter), but consistently across all of the adapters that could benefit.