Plots2: Image upload failing

Created on 4 Apr 2021  ·  35Comments  ·  Source: publiclab/plots2

Image upload fails with this error in the body section "Section 4" of /post route
Screenshot from 2021-04-04 03-28-07 this is on the https://publiclab.org/, was able to replicate it on https://unstable.publiclab.org and locally

Note: Image upload is working fine on "Section 2" of /post
Template: https://github.com/publiclab/plots2/blob/main/app/views/editor/rich.html.erb

bug help wanted high-priority

Most helpful comment

OK, @icarito and I were able to resolve a yarn update issue to pull in the real 3.0.3 editor code, confirmed it, then fixed one more nginx config issue related to SSL and it is now working on the live site. Thanks, everyone!

image

All 35 comments

Also referencing #9442 for a minor problem.

thanks @waridrox :+1:

Attempting to debug the problem further, the error that I got when uploading an image from local file storage was
Paperclip::Errors::CommandNotFoundError: Could not run the 'identify' command. Please install ImageMagick.
under local dev environment. This I believe is a new dependency for paperclip gem that deals with file uploads.

Screenshot 2021-04-04 at 8 13 15 PM

After installing imagemagick with the command sudo apt-get install imagemagick or brew install imagemagick, I added these 2 lines of code in development.rb file.

Paperclip.options[:image_magick_path] = "/opt/ImageMagick/bin"
Paperclip.options[:command_path] = "/opt/ImageMagick/bin"

After that I simply restarted the server and the file upload was functional again with file types like jpeg, png and gif. Here's a glimpse of the same -

https://user-images.githubusercontent.com/58583793/113514702-c308e700-958d-11eb-8e12-59c7a9648b24.mp4

From _this thread_ on stackoverflow, we need to add these 2 lines in production.rb as well to reflect the changes in the production app.

Also due to optional parameter to install image-magick in https://github.com/publiclab/plots2/blob/main/doc/PREREQUISITES.md#image-libraries-optional, this behaviour was observed in the first place.

Thanks @waridrox, I pulled your changes and I am still getting the error....could someone else please try on their end. Thanks

Oh 😅 really sorry about that, pls revert the pull then. Does it work locally though ? Maybe trying sudo apt-get update first and then sudo apt-get install imagemagick can help...

Hi all, locally it makes some sense that it's an imagemagick but (or absence of imagemagick) but on stable it should be installed. I wonder if it's a distinct bug on stable? We should cross check on Sentry.io. Cess do you have access?

Thanks for finding and documenting this!!

I'm not seeing sentry errors on image upload except in relation to profile pics (which is often the case due to spam and attacks).

Screenshot_20210404-174638
Screenshot_20210404-174706

What is the path for the error? It's a 500 error?

Sorry I mean /recent/ errors

@waridrox it didn't work for me locally maybe I have the wrong version..@jywarren it was more of a 302 error on my end

Screenshot from 2021-04-05 00-54-02

Aha, 302 error indeed... 🕵️

I'll look at why that may be the case. Thank you!

I'm going to try running it in GitPod to see if i can reproduce the 302...

OK, got a log from GitPod!

Started POST "/images" for 10.4.6.182 at 2021-04-13 18:36:40 +0000
Cannot render console from 10.4.0.249! Allowed networks: 127.0.0.1, ::1, 127.0.0.0/127.255.255.255
Processing by ImagesController#create as JSON
  Parameters: {"nid"=>"null", "image"=>{"photo"=>#<ActionDispatch::Http::UploadedFile:0x00007f444ce23c30 @tempfile=#<Tempfile:/tmp/RackMultipart20210413-4879-sla487.jpg>, @original_filename="11aa-love-ryan2-720.jpg", @content_type="image/jpeg", @headers="Content-Disposition: form-data; name=\"image[photo]\"; filename=\"11aa-love-ryan2-720.jpg\"\r\nContent-Type: image/jpeg\r\n">}}
Can't verify CSRF token authenticity.
Redirected to https://3000-aquamarine-bass-qsunlo8p.ws-us03.gitpod.io/login?return_to=/images
Filter chain halted as :require_user rendered or redirected
Completed 302 Found in 2ms (ActiveRecord: 0.0ms)


Started GET "/login?return_to=/images" for 10.4.6.182 at 2021-04-13 18:36:40 +0000
Cannot render console from 10.4.0.249! Allowed networks: 127.0.0.1, ::1, 127.0.0.0/127.255.255.255
Processing by UserSessionsController#new as JSON
  Parameters: {"return_to"=>"/images"}
  User Load (0.5ms)  SELECT  `rusers`.* FROM `rusers` WHERE `rusers`.`id` = 1 LIMIT 1
   (0.2ms)  BEGIN
  User Update (0.5ms)  UPDATE `rusers` SET `last_request_at` = '2021-04-13 18:36:40', `updated_at` = '2021-04-13 18:36:40' WHERE `rusers`.`id` = 1
   (28.6ms)  COMMIT
Redirected to https://localhost/home?return_to=%2Flogin
Filter chain halted as :require_no_user rendered or redirected
Completed 302 Found in 35ms (ActiveRecord: 29.7ms)

OK, so why are we being redirected to log in... looking...

Looks like CSRF token is not working... let's see...

https://github.com/publiclab/plots2/blob/bd7b2e26c1edbd85a9754d8b25bc27950fd99770/app/views/editor/rich.html.erb#L344-L347

we pass it in here.

$('meta[name="csrf-token"]').attr('content') correctly fetches the token, so that's not it...

Noting that in GitPod, the main image uploader works. It uses the same controller, so the issue seem likely to be in the Editor's rich text module code itself:


Started POST "/images" for 10.4.0.248 at 2021-04-13 18:47:45 +0000
Cannot render console from 10.4.6.181! Allowed networks: 127.0.0.1, ::1, 127.0.0.0/127.255.255.255
Processing by ImagesController#create as JSON
  Parameters: {"authenticity_token"=>"5Z5plBlzFXbu2qCKPTDZYa54DWaNn556hs9FzyvH2U8NDMabePLVh1RlBuhkRR+Ej+kgI/hLIAm9LcdrEZv4lg==", "uid"=>"1", "image"=>{"photo"=>#<ActionDispatch::Http::UploadedFile:0x000055644bef1898 @tempfile=#<Tempfile:/tmp/RackMultipart20210413-5664-bsht8f.jpg>, @original_filename="11aa-love-ryan2-720.jpg", @content_type="image/jpeg", @headers="Content-Disposition: form-data; name=\"image[photo]\"; filename=\"11aa-love-ryan2-720.jpg\"\r\nContent-Type: image/jpeg\r\n">}}

And actually the main image uploader also uses the token in its request:

https://github.com/publiclab/PublicLab.Editor/blob/59e5bf6fd1ac2b6f9108e522baa7cdab36e0bb64/src/modules/PublicLab.MainImageModule.js#L94

It correctly fetches it with:

editor.options.mainImageModule.token
"5Z5plBlzFXbu2qCKPTDZYa54DWaNn556hs9FzyvH2U8NDMabePLVh1RlBuhkRR+Ej+kgI/hLIAm9LcdrEZv4lg=="

Confirmed. The main image upload has this set of params in the form request:

authenticity_token: 5Z5plBlzFXbu2qCKPTDZYa54DWaNn556hs9FzyvH2U8NDMabePLVh1RlBuhkRR+Ej+kgI/hLIAm9LcdrEZv4lg==
uid: 1
image[photo]: (binary)

Whereas the inline image upload has only these:

nid: null
image[photo]: (binary)

checking recent changes to that section of code, as well as the module options itself to ensure it's getting properly stored/passed in from the constructor.

Wow, deep bug. Chasing back through https://github.com/jywarren/woofmark/pull/2 to woofmark library: https://github.com/bevacqua/woofmark/pull/44

I believe this PR from 28 days ago may be related. the last change to the branch of woofmark we are using happened in September 2020.

https://github.com/jywarren/woofmark/pull/76/files

I'm not seeing anything in the code in woofmark -- 2 routes now:

  1. tracing the upload.xhrOptions param through Woofmark and confirming it's correctly got it
  2. seeing if the xhr npm module has changed

https://www.npmjs.com/package/xhr published 2.6 5 months ago but has been at 2.2.1 for many many months in our projects. confirmed in package-lock.json

OK, so the upload.xhrOptions param beforeSend does make it into woofmark, as tested in the JS console.

Does it actually generate the extra header?

xhrOptions: { 
        beforeSend: function(xhr) { xhr.setRequestHeader('X-CSRF-Token', $('meta[name="csrf-token"]').attr('content')) }
      },

Gosh i don't know when this stopped working but I think we could potentially just add another formData param and do it that way??? But not sure if it'll get passed up with the others, or if the xhr lib will ignore it?

Noting this would have happened either November 5 2020 in https://github.com/publiclab/PublicLab.Editor/releases/tag/v3.0 or 28 days ago in https://github.com/publiclab/plots2/pull/9323

According to this line, i believe formData should get passed in normally:

https://github.com/bevacqua/woofmark/pull/44/files#diff-b70752c6e4fb751c6aa381f57afac66c25cc2d401b141b03e1751d81e60efcd2R216

will try in GitPod:

      formData: {nid: null, authenticity_token: _module.options.token},

... THAT DID IT:

image

Will fix in https://github.com/publiclab/PublicLab.Editor/pull/712, release 3.0.3, and also add the extra formData token in https://github.com/publiclab/plots2/pull/9504.

Whew!

Done and confirmed in stable!

image

@publiclab-mimi reported today that

On the research note editor I was unable to upload a JPEG, PDF, or PNG.
On Google Chrome and Safari both drag-and-drop as well as browse options

@ebarry, I think the current version still doesn't have the above changes due to buffer time required to push the changes to the live version. That's because image / file upload works on the stable version and the legacy version.

Screenshot 2021-04-20 at 2 54 03 AM

The new code went live last night! Taking a look, i'm seeing a new error - a 500 error:

starting with this one, which is the same 302 as Cess found:

image

Then redirected to this new 500 error:

image

which is actually upon a successful login, i believe, but it's weird because i'm already logged in?

Sentry logged it here, i think, but i'm not sure if the error happens before or after the login attempt.

https://sentry.io/share/issue/13e10e65210c4ceb9860ec687482d9b7/

```
ActionView::MissingTemplate
Missing template home/home with {:locale=>[:en], :formats=>[:json], :variants=>[], :handlers=>[:raw, :erb, :html, :builder, :ruby, :coffee, :jbuilder]}. Searched in:

  • "/app/app/views"
  • "/usr/local/bundle/gems/grape-swagger-ui-2.2.8/app/views"
  • "/usr/local/bundle/gems/grape-swagger-rails-0.3.1/app/views"
  • ```

Huh, having trouble tracing this so i'll try reproducing in GitPod again???

OK, @icarito and I were able to resolve a yarn update issue to pull in the real 3.0.3 editor code, confirmed it, then fixed one more nginx config issue related to SSL and it is now working on the live site. Thanks, everyone!

image

:tada: :tada: :tada:

awesome!!! thank you

Was this page helpful?
0 / 5 - 0 ratings

Related issues

first-timers[bot] picture first-timers[bot]  ·  3Comments

divyabaid16 picture divyabaid16  ·  3Comments

milaaraujo picture milaaraujo  ·  3Comments

keshavsethi picture keshavsethi  ·  3Comments

grvsachdeva picture grvsachdeva  ·  3Comments