Gsutil: Cannot execute a sequential gsutil in windows batch file

Hi - did the first command exit with a non-0 exit status? That would cause the batch file to abort.

I've tested like below:

a.exe
echo Exit Code is %errorlevel%
gsutil cp -R a gs://zelon-test/
echo Exit Code is %errorlevel%
gsutil cp -R b gs://zelon-test/
echo Exit Code is %errorlevel%

below is the result:

C:\Temp>test.bat

C:\Temp>a.exe

C:\Temp>echo Exit Code is 0
Exit Code is 0

C:\Temp>gsutil cp -R a gs://zelon-test/
Copying file://a\find.txt [Content-Type=text/plain]...
Uploading gs://zelon-test/a/find.txt: 36 KB/36 KB

C:\Temp>

First echo command is executed and show the return value of a.exe. But second echo command(after first gsutil command) is not executed. It seems that gsutil kill the windows batch process.

I just tried running the following script on Windows 7 Ultimate with Service Pack 1, and it worked for me:

\Python27\python.exe \cygwin\usrlocal\bin\gsutil\gsutil cp c:\try.bat gs://my-bucket/1
\Python27\python.exe \cygwin\usrlocal\bin\gsutil\gsutil cp c:\try.bat gs://my-bucket/2

What version of Windows are you runing?
What happens if you change the commands to run gsutil using the full path to the python interpreter and to the gsutil script, similar to what I did above?

Mike

I've address this issue with your advice. Thank you.

I'm using Windows 8.1 Pro K(Korean) and default python version is 2.7.6 64 bit
And I tried to use python with full path as you mentioned. Then, some error message appeared like below:

Uploading gs://zelon-test/a.exe: 14 KB/14 KB
ERROR 1108 16:48:44.817000 http_wrapper.py] Response returned status 401, retrying
ERROR 1108 16:48:44.817000 http_wrapper.py] Retrying request to url https://www.googleapis.com/upload/XXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXuploadType=multipart after exception HttpError accessing fields=generation%2Ccrc32c%2Cmd5Hash%2Csize&alt=json&key=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX&prettyPrint=True&upload
Type=multipart>: response: <{'status': '401', 'alternate-protocol': '443:quic,p=0.01', 'content-length': '238', 'vary':
'Origin, Referer, X-Origin', 'server': 'UploadServer ("Built on Oct 29 2014 16:52:36 (1414626756)")', 'date': 'Sat, 08 N
ov 2014 07:48:45 GMT', 'content-type': 'application/json; charset=UTF-8', 'www-authenticate': 'Bearer realm="https://acc
ounts.google.com/AuthSubRequest"'}>, content <{
"error": {
"errors": [
{
"domain": "global",
"reason": "required",
"message": "Login Required",
"locationType": "header",
"location": "Authorization"
}
],
"code": 401,
"message": "Login Required"
}
}

So I used "gcloud auth login" and I can use test.bat correctly for now.

With short form like "gsutil", I still cannot use that test.bat. With full path like "C:\Python27\python.exe E:\Programs\google-cloud-sdk\platform\gsutil\gsutil.py" I can use that test.bat

Hi - I've never been able to get gsutil to work on Windows without fully specified paths - that's how we recommend you use gsutil on Windows (see the Windows tabs of the various pieces of documentation at https://cloud.google.com/storage/docs/gsutil_install).

I'm closing this issue because it's working as documented. I'm glad you were able to get things working.

Thanks. This has helped me too.

I just stumbled upon this problem myself (I run several gcloud commands to transfer data periodically). In your batch file be sure to preface gcloud or gsutil or such with call, thus:

call gcloud compute copy-files foobarhost:/home/foobar/blah /my/local/folder --zone my-compute-zone
call gcloud compute copy-files foobarhost:/home/foobar/blah /my/local/folder --zone my-compute-zone
...etc...

Using call prevents the batch from exiting as soon as its child batch exits. You can use %ERRORLEVEL% to check the result of the called command (the GCP scripts set this appropriately). See also http://ss64.com/nt/call.html

THAT is the answer! Thank you Marty!

"There always seems to be a way to fix things." - MacGyver

Indeed, thanks Marty, I didn't know about that either.

i have the same exact problem but when i use the full paths, i am using the paths: "C:\Python27\python.exe" "C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\platformgsutilgsutil.py" cp %myfile.txt% gs://%myBucket%/%folder%/ getting the exception:

ServiceException: 401 Anonymous caller does not have storage.objects.create access to %myBucket%/%folder%/%myfile.txt%

this does not happen when i use the gsutil comand, Any recomendation?

i have the same exact problem but when i use the full paths, i am using the paths: "C:\Python27\python.exe" "C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\platformgsutilgsutil.py" cp %myfile.txt% gs://%myBucket%/%folder%/ getting the exception:

ServiceException: 401 Anonymous caller does not have storage.objects.create access to %myBucket%/%folder%/%myfile.txt%

this does not happen when i use the gsutil comand, Any recomendation?

UPDATE:

i manage to work around the problem creating a new process for every operation as child's of the original batch file process using the call comand. therefore the comands inside the batch files now look something like this:

* call cmd /c gsutil cp %file_1% gs://%bucket%/%folder%/ *
* call cmd /c gsutil cp %file_2% gs://%bucket%/%folder%/ *

note that the /c allows the original batch process to wait for the other sub-process so all the operation looks just as it should be