Gsutil: Option to disable progress spinner

Created on 27 Oct 2017 · 9Comments · Source: GoogleCloudPlatform/gsutil

I am using gsutil in a scripting environment to move large files. I need to have progress logged so we can monitor jobs and get status updates. However, the progress indicator for gsutil cp includes a spinner that assumes it is working in a terminal. This produces an incredible amount of garbage in a log file.

This is related to #429 and #265 but is not the same request. I still need to see the progress, it just needs to omit the special control characters that assume the output is a terminal window. In other words, I want an option to remove the spinner and instead just write a new line.

Feature Request

Source

kent-at-multiscale

👍23 😄1

All 9 comments

Thanks for submitting the request -- I've seen a few mentions of this since we did the UI overhaul and added the spinner-based progress printing. I agree that it would be much nicer to provide a way for gsutil to do progress printing that doesn't look like a mess when not directed to a terminal, either by providing an explicit option for this or (probably better) to have gsutil automatically do this if stdout refers to something besides a terminal.

houglum on 28 Oct 2017

👍1

I second a need for this. We're running gsutil through Ant, and don't see any progress updates until the command finishes, whereupon all updates are concatenated. Can't see any workaround for this using Ant options. Ideally we'd see a newline per progress update, every minute perhaps. Some option not to be clever with the terminal? Rather than disabling altogether via -q.

supersteves on 6 Nov 2017

Plus one. Having to choose between reams of garbage directed to log file or complete silence is not ideal. Thanks!

keeth on 20 Feb 2018

Plus one! :)

berlincount on 16 Mar 2018

dantheperson on 12 Oct 2018

We already have a helper function IsRunningInteractively() which tests whether stdin/out isatty():
https://github.com/GoogleCloudPlatform/gsutil/blob/e2f02ab4d683cb509c847e1b452a75ce526fe83a/gslib/utils/system_util.py#L233-L235

The StatusMessageManager should call IsRunningInteractively() at some point to decide whether to actually show the spinner:
https://github.com/GoogleCloudPlatform/gsutil/blob/e2f02ab4d683cb509c847e1b452a75ce526fe83a/gslib/ui_controller.py#L114-L117

jeremyvisser on 18 Mar 2019

+1
Not sure if we're talking about the same thing here but when I run my PS script from Task Scheduler I see this kind of garbage in my log when using gsutil cp ... :

Uploading gs://temp/db1.zip: 0 B/1.20 GiB
Uploading gs://temp/db1.zip: 98.75 MiB/1.20
Uploading gs://temp/db1.zip: 98.81 MiB/1.20
...and keeps repeating this until it uploads the entire 1.20 GB.

This can be tens or hundreds of lines depending on the file size. When I run the same script from a PS command prompt the progress is displayed on line.

_Source: https://stackoverflow.com/questions/55654436/running-powershell-script-with-gsutil-cp-to-gcp-shows-upload-progress-on-multipl/55669118#55669118_

hero2017 on 14 Apr 2019

👍1

Our Jenkins output has a few extra thousands of lines with:

/ [274/418 files][454.8 MiB/  1.2 GiB]  35% Done                                
/ [275/418 files][455.9 MiB/  1.2 GiB]  35% Done                                
/ [276/418 files][456.6 MiB/  1.2 GiB]  36% Done                                
/ [277/418 files][457.9 MiB/  1.2 GiB]  36% Done                                
/ [278/418 files][457.9 MiB/  1.2 GiB]  36% Done                                
/ [279/418 files][457.9 MiB/  1.2 GiB]  36% Done

-q - option will hide progress and filenames too. It is more convenient to have file names and status of the downloading in output but disable printing of the progress bar.