Azure-sdk-for-java: Azure Blob - listBlobsByHierarchy().iterableByPage(10) - not providing results based on defined page size

Created on 19 Mar 2021 · 3Comments · Source: Azure/azure-sdk-for-java

I have a folder in blob container which have 112 blobs. I defined to get 10 blobs per page, but I am getting all 112 blobs in one page response. Please find below sample code to reproduce this issue.

PagedIterable<BlobItem> pagedIterable = getBlobContainerClient().listBlobsByHierarchy("batch/");

        Iterable<PagedResponse<BlobItem>> iterableByPage = pagedIterable.iterableByPage(10);

        for (PagedResponse<BlobItem> pagedResponse : iterableByPage) {
            Iterator<BlobItem> iterableStream = pagedResponse.getElements().iterator();
            int counter = 1;
            System.err.println("counter - " + counter);

            while (iterableStream.hasNext()) {
                BlobItem blobItem = iterableStream.next();
                LOGGER.info(counter++ +" ** listBlob name -'{}' properties-'{}', tags -'{}', version -'{}', snapshot -'{}' ***", 
                                        blobItem.getName(), blobItem.getProperties(), blobItem.getTags(), blobItem.getVersionId(),
                                        blobItem.getSnapshot());

            }
        }

ScreenShot

To Reproduce

Create Folder with 112 items
Run above sample code and you will get all items in a single page response

Expected behavior
Only 10 blobs should be available in one page response.

Setup (please complete the following information if applicable):
OS: window
IDE : eclipse
Version of the Library used:azure-storage-blob-12.10.0

Client Storage bug customer-reported

Source

abhikt48

All 3 comments

Hi @abhikt48
Thank you for posting this issue. We will take a look at what is going wrong here.

gapra-msft on 19 Mar 2021

👍1

This looks to be a duplicate of #12496 As Gauri mentioned, I'll look into reproducing this either today or Monday

rickle-msft on 19 Mar 2021

👍1

@rickle-msft Just FYI, I did same testing with another approach and it is working fine. I provided custom ListBlobsOptions and setted setMaxResultsPerPage. But I didn't find any affect of pagedIterable.iterableByPage(10), looks like this is redundant. Could you please check and confirm ?

ListBlobsOptions listBlobsOptions = new ListBlobsOptions();
        listBlobsOptions.setPrefix(directoryName);
        listBlobsOptions.setMaxResultsPerPage(10);        
PagedIterable<BlobItem> pagedIterable = getBlobContainerClient().listBlobsByHierarchy(null, listBlobsOptions , null);
Iterable<PagedResponse<BlobItem>> iterableByPage = pagedIterable.iterableByPage(10);

Could you please help me with below queries related with listBlobsByHierarchy(null, listBlobsOptions , null) operation -

listBlobsOptions.setPrefix(directoryName + "prefix"); - I didn't find any option to set directory specifically in ListBlobsOptions or BlobContainerClient.listBlobsByHierarchy(null, listBlobsOptions , null). Can I assume that - directoryName can be provide via listBlobsOptions.setPrefix() only ? Please confirm.
Old SDK provides option to get length of result, but I didn't find any option in New SDK with listBlobsByHierarchy opertaion. Could you please check and confirm if we have any option to check size of list

Old SDK - Length of list

ResultSegment<ListBlobItem> listBlobResult = provider.getBlobListInDirectory(containerName, directoryName, 
                                                                blobPrefix, index, continuationToken);
        totalItems = listBlobResult.getLength();

Please suggest your opinion on above queries, so i can proceed futher.

Regards,
Abhishek Kumar

abhikt48 on 22 Mar 2021

Was this page helpful?

0 / 5 - 0 ratings