Office365-rest-python-client: How to access remote files on the server with special characters in their names?

I am using your Python client to download files from a SharePoint folder. That basically works, however, some files have uncommon names (with special characters) and with those the download does not work. For example, a typical such file name would be Sequenz%2001.mp4#t=0.033333.jpg - that is not encoded, the file name is exactly that.

When I try to access that file under this name, then I get a "file not found" error. But when I rename the file to test.jpg then the download works. I am following your example to download the files:

FILE = "/Sequenz%2001.mp4#t=0.033333.jpg"  # it does work with "test.jpg" here
# Some more constants for USER, PASS, URL, etc.

ctx_auth = AuthenticationContext(URL)
if ctx_auth.acquire_token_for_user(USER, PASS):
    ctx = ClientContext(URL + SITE, ctx_auth)
    local_file_name = "/tmp" + FILE
    try:
        with open(local_file_name, "wb") as local_file:
            f = ctx.web.get_file_by_server_relative_url(SITE + PATH + FILE)
            f.download(local_file).execute_query()
    except ClientRequestException:
        print("Can't fetch remote file: " + PATH + FILE)

I tried to quote the file name via urllib.parse.quote() and use that, but with the same result.
What do I need to access such a file?