Faraday: Set array parameters from a Ruby array

I'm a little torn on this — the convention of using bar[] is one that comes from rails (and was adopted by rack).

The CGI specification supports what faraday currently does (as does the ruby CGI library).

If we make the change, I'm not sure how you would be able to pass them the way it is now...

Why would you ever want to do that? It's akin to creating a Ruby hash like this:

{'bar' => 'baz', 'bar' => 'qux'}

Sure, it's valid syntax, but the resulting hash will only have one value, so for all intents and purposes, it's pointless.

Honestly, there's nothing different as far as the basic parsing goes between having two fields in the query string that are bar[] vs bar. The [] suffix is a convention that Rails came up with to make it so that the consumer of the hash of query params didn't have to worry if a given key would return a single element or an array.

If you look at the ruby CGI library, you can see it actually returns an array for parsing two bars:

>> CGI.parse('bar=baz&bar=qux')
=> {"bar"=>["baz", "qux"]}

Here's more info on it: http://en.wikipedia.org/wiki/Query_string

Unfortunately making Faraday adhere to this convention that Rails came up with could make it far more difficult to interact with some other kinds of HTTP endpoints.

One solution to this may be to write some simple Faraday request middleware that does this fix-up for you.

I see your point. I suppose I lean towards Faraday mimicking Rack's behavior to be consistent with the rest of the library.

That makes sense for Ruby frameworks but doesn't make sense for some other frameworks that parse out the current scheme correctly. I'm not sure what approach should we take here, but since we're in ruby world I think we should go your way and serialize arrays by appending "[]" to keys.

I'm definitely concerned that these sorts of things happening behind the scenes causing confusion for users...

Yes @eric might be right. We could leave it as is and encourage users to post JSON to Ruby applications? That way they don't need to worry about serialization conventions.

We could leave it as is and encourage users to post JSON to Ruby applications?

Ha. You're funny.

I ran into this issue while implementing a new method in the Twitter API. Do you think I'll have much luck convincing them?

I would add that POST is not the appropriate verb for every HTTP request. Would you suggest URL-encoding JSON objects and adding them to the query string for GET requests? Something like:

GET /foo?body=%7B%22bar%22%3A%5B%22baz%22%2C%22qux%22%5D%7D

Dear god, I hope not.

I totally think it would make sense for us to create a standard-ish middleware that you can insert if you want the rails-style encoding.

I'd be okay with that solution, though I'm a bit worried we're wielding middleware as Maslow's hammer.

Implementation aside, I'd like to make the case that this should be the default behavior, since it seems like the more common use case and is more in-line with this project's overall design.

By the way, this isn't just a Ruby/Rails thing. The jQuery.param() method serializes arrays with brackets as of version 1.4.

$.param({bar: ['baz', 'qux']}) // "bar[]=baz&bar[]=qux"

It appears that this is also the default behavior in PHP. In fact, I'd bet the convention existed in one or more PHP frameworks before it existed in Rails.

You're right, @sferik, I forgot about GET params.

I vote to change the behavior to add brackets since it seems to be prevalent in our wider community. The UrlEncoded middleware should handle this. This middleware could also have the option to turn it off.

conn.request :url_encoded, bracketize: false

Suggestions how to call this option welcome.

I have been summoned! Heh.

RFC 3986 covers URIs. Section 3.4 covers query strings in URIs. I don't see anything in particular covering duplicate keys, or a convention on how to handle them. So this is largely an aesthetic decision.

Personally, I like the []s.

You have any thoughts, @technoweenie?

@sferik: That is very interesting that jQuery is following that convention as well. I would love to see the origin of it. I don't believe I had seen it anywhere before Rails.

@mislav: I could see that...

Everyone: Is it really uncomfortable to have to specify "bar[]" or is it the asymmetry that is bothersome?

As long as we make a point to document the option very clearly, I would feel less queazy about it. I'm super concerned about these sorts of things that do magic under the covers that require people to dig deeply to find a solution to behavior that is confusing to them (if they are coming from a different perspective).

My main concern is about having the right default behavior.

I found myself in the position of needing to generate a URL that looked like:

GET /foo?bar[]=baz&bar[]=qux

My first guess for how to do that was:

request.url('foo', 'bar' => ['baz', 'qux'])

In general, one of my favorite things about Ruby is that my first guess on how to do something will often "just work". Since Ruby has such flexible syntax, our community places more emphasis on making interfaces as intuitive as possible. In this case, my intuition failed me, so I opened this issue to see if others shared the same intuition about how it _should_ work.

I like @mislav's proposal to enable this behavior by default in UrlEncoded middleware.

I agree that it should be possible to disable. I also agree that it should be clearly documented.

And I, too, would be interested to hear @technoweenie's take on the issue before moving forward, however, he's in the middle of moving right now, so we may need to wait a few days. This is certainly not an urgent issue. Just a minor bugaboo.

I'd say go with brackets by default. Don't be like ASP!

Request.QueryString("chkFoo")(1)

I think doing it in the middleware with a simple way to override the default on a per Connection basis is the right way to go.

Okay. Looks like I lost. As long as there's a way to override it to be able to interact with other services that don't use the [] notation.

Bump. UrlEncoded middleware uses Rack's build_nested_query for serializing params in request bodies, even after we removed the Rack dependency. It adds the [] to array keys. @sferik opened this discussion to GET params which are serialized using build_query, which doesn't add [] to array keys nor handle nested params.

So, the first problem is we serialize params in POST bodies differently than GET query strings. We should address this inconsistency by switching from build_query to build_nested_query for GET params.

The second problem is array keys. After we're consistent, they will automatically get suffixed with []. To turn this off, we suggested a per-request option (which can also be configured per-connection) such as {bracketize: false}. However implementing this is not as trivial for GET params as it is for POST params, because GET params get serialized in Connection#build_url while POST params get serialized in UrlEncoded middleware. So, two entirely different places.

It's not impossible, though. We can pass request options to each invocation of build_url to make it aware of the new setting.

This has come up again as #182.

Nothing against _supporting_ the [] convention, but definitely needs a way to turn it off. Not every web service out there use the convention, and latest version of Faraday makes it impossible to call these.

@sqrrrl Precisely.

@sferik Let's say I find myself needing to generate a URL like:

GET /foo?bar=baz&bar=qux

My first guess for how to do that would also be:

request.url('foo', 'bar' => ['baz', 'qux'])

So I'm not sure that's really an argument in the square-bracket-notation's favor.

And I think there's an argument that could be made that some people might actually try this first anyways in your example scenario:

request.url('foo', 'bar[]' => ['baz', 'qux'])

Given that pull request #199 was merged, this can probably be closed.

The pull-request is a bit vague, would be good to add a quick example when closing such an issue :+1:

Sorry I was a bit in a rush and couldn't provide an example here, but you can see a good example from the tests in #199:

def test_build_url_bracketizes_repeated_params_in_query
  conn = Faraday::Connection.new
  uri = conn.build_url("http://sushi.com/sake.html", 'a' => [1, 2])
  assert_equal "a%5B%5D=1&a%5B%5D=2", uri.query
end

def test_build_url_without_braketizing_repeated_params_in_query
  conn = Faraday::Connection.new
  conn.options[:params_encoder] = Faraday::FlatParamsEncoder
  uri = conn.build_url("http://sushi.com/sake.html", 'a' => [1, 2])
  assert_equal "a=1&a=2", uri.query
end

In short, Faraday now by default serialises array "the rack way" with the "[]" after the variable name.
However, you can set Faraday::FlatParamsEncoder as params_encoder to get them serialise without brackets.

Hi I am facing the same issue when I am sending parameters with array to faraday request. It is appending extra [] brackets to the key. I tried appending params_encoder attribute to faraday request but getting error like undefined method params_encoder

I am using faraday 0.9.2 version with rails 5.

Can you please help to fix on this

Hi @anushamummina, version 0.9.2 should already contain all the changes to let you set params_encoder. Are you able to provide a code snippet to show the error you're getting?