• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    公众号

Java BlobMetaData类代码示例

原作者: [db:作者] 来自: [db:来源] 收藏 邀请

本文整理汇总了Java中org.elasticsearch.common.blobstore.BlobMetaData的典型用法代码示例。如果您正苦于以下问题:Java BlobMetaData类的具体用法?Java BlobMetaData怎么用?Java BlobMetaData使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。



BlobMetaData类属于org.elasticsearch.common.blobstore包,在下文中一共展示了BlobMetaData类的20个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: listBlobsToGetLatestIndexId

import org.elasticsearch.common.blobstore.BlobMetaData; //导入依赖的package包/类
private long listBlobsToGetLatestIndexId() throws IOException {
    Map<String, BlobMetaData> blobs = snapshotsBlobContainer.listBlobsByPrefix(INDEX_FILE_PREFIX);
    long latest = RepositoryData.EMPTY_REPO_GEN;
    if (blobs.isEmpty()) {
        // no snapshot index blobs have been written yet
        return latest;
    }
    for (final BlobMetaData blobMetaData : blobs.values()) {
        final String blobName = blobMetaData.name();
        try {
            final long curr = Long.parseLong(blobName.substring(INDEX_FILE_PREFIX.length()));
            latest = Math.max(latest, curr);
        } catch (NumberFormatException nfe) {
            // the index- blob wasn't of the format index-N where N is a number,
            // no idea what this blob is but it doesn't belong in the repository!
            logger.debug("[{}] Unknown blob in the repository: {}", metadata.name(), blobName);
        }
    }
    return latest;
}
 
开发者ID:justor,项目名称:elasticsearch_my,代码行数:21,代码来源:BlobStoreRepository.java


示例2: findLatestFileNameGeneration

import org.elasticsearch.common.blobstore.BlobMetaData; //导入依赖的package包/类
/**
 * Finds the next available blob number
 *
 * @param blobs list of blobs in the repository
 * @return next available blob number
 */
protected long findLatestFileNameGeneration(Map<String, BlobMetaData> blobs) {
    long generation = -1;
    for (String name : blobs.keySet()) {
        if (!name.startsWith(DATA_BLOB_PREFIX)) {
            continue;
        }
        name = BlobStoreIndexShardSnapshot.FileInfo.canonicalName(name);
        try {
            long currentGen = Long.parseLong(name.substring(DATA_BLOB_PREFIX.length()), Character.MAX_RADIX);
            if (currentGen > generation) {
                generation = currentGen;
            }
        } catch (NumberFormatException e) {
            logger.warn("file [{}] does not conform to the '{}' schema", name, DATA_BLOB_PREFIX);
        }
    }
    return generation;
}
 
开发者ID:justor,项目名称:elasticsearch_my,代码行数:25,代码来源:BlobStoreRepository.java


示例3: snapshotFileExistsInBlobs

import org.elasticsearch.common.blobstore.BlobMetaData; //导入依赖的package包/类
/**
 * Checks if snapshot file already exists in the list of blobs
 *
 * @param fileInfo file to check
 * @param blobs    list of blobs
 * @return true if file exists in the list of blobs
 */
private boolean snapshotFileExistsInBlobs(BlobStoreIndexShardSnapshot.FileInfo fileInfo, Map<String, BlobMetaData> blobs) {
    BlobMetaData blobMetaData = blobs.get(fileInfo.name());
    if (blobMetaData != null) {
        return blobMetaData.length() == fileInfo.length();
    } else if (blobs.containsKey(fileInfo.partName(0))) {
        // multi part file sum up the size and check
        int part = 0;
        long totalSize = 0;
        while (true) {
            blobMetaData = blobs.get(fileInfo.partName(part++));
            if (blobMetaData == null) {
                break;
            }
            totalSize += blobMetaData.length();
        }
        return totalSize == fileInfo.length();
    }
    // no file, not exact and not multipart
    return false;
}
 
开发者ID:justor,项目名称:elasticsearch_my,代码行数:28,代码来源:BlobStoreRepository.java


示例4: listBlobsByPrefix

import org.elasticsearch.common.blobstore.BlobMetaData; //导入依赖的package包/类
@Override
public Map<String, BlobMetaData> listBlobsByPrefix(String blobNamePrefix) throws IOException {
    // If we get duplicate files we should just take the last entry
    Map<String, BlobMetaData> builder = new HashMap<>();

    blobNamePrefix = blobNamePrefix == null ? "" : blobNamePrefix;
    try (DirectoryStream<Path> stream = Files.newDirectoryStream(path, blobNamePrefix + "*")) {
        for (Path file : stream) {
            final BasicFileAttributes attrs = Files.readAttributes(file, BasicFileAttributes.class);
            if (attrs.isRegularFile()) {
                builder.put(file.getFileName().toString(), new PlainBlobMetaData(file.getFileName().toString(), attrs.size()));
            }
        }
    }
    return unmodifiableMap(builder);
}
 
开发者ID:justor,项目名称:elasticsearch_my,代码行数:17,代码来源:FsBlobContainer.java


示例5: testCompressionIsApplied

import org.elasticsearch.common.blobstore.BlobMetaData; //导入依赖的package包/类
public void testCompressionIsApplied() throws IOException {
    BlobStore blobStore = createTestBlobStore();
    BlobContainer blobContainer = blobStore.blobContainer(BlobPath.cleanPath());
    StringBuilder veryRedundantText = new StringBuilder();
    for (int i = 0; i < randomIntBetween(100, 300); i++) {
        veryRedundantText.append("Blah ");
    }
    ChecksumBlobStoreFormat<BlobObj> checksumFormat = new ChecksumBlobStoreFormat<>(BLOB_CODEC, "%s", BlobObj::fromXContent,
        xContentRegistry(), false, randomBoolean() ? XContentType.SMILE : XContentType.JSON);
    ChecksumBlobStoreFormat<BlobObj> checksumFormatComp = new ChecksumBlobStoreFormat<>(BLOB_CODEC, "%s", BlobObj::fromXContent,
        xContentRegistry(), true, randomBoolean() ? XContentType.SMILE : XContentType.JSON);
    BlobObj blobObj = new BlobObj(veryRedundantText.toString());
    checksumFormatComp.write(blobObj, blobContainer, "blob-comp");
    checksumFormat.write(blobObj, blobContainer, "blob-not-comp");
    Map<String, BlobMetaData> blobs = blobContainer.listBlobsByPrefix("blob-");
    assertEquals(blobs.size(), 2);
    assertThat(blobs.get("blob-not-comp").length(), greaterThan(blobs.get("blob-comp").length()));
}
 
开发者ID:justor,项目名称:elasticsearch_my,代码行数:19,代码来源:BlobStoreFormatIT.java


示例6: listBlobsByPrefix

import org.elasticsearch.common.blobstore.BlobMetaData; //导入依赖的package包/类
@Override
public Map<String, BlobMetaData> listBlobsByPrefix(String account, LocationMode mode, String container, String keyPath, String prefix) throws URISyntaxException, StorageException {
    // NOTE: this should be here: if (prefix == null) prefix = "";
    // however, this is really inefficient since deleteBlobsByPrefix enumerates everything and
    // then does a prefix match on the result; it should just call listBlobsByPrefix with the prefix!

    logger.debug("listing container [{}], keyPath [{}], prefix [{}]", container, keyPath, prefix);
    MapBuilder<String, BlobMetaData> blobsBuilder = MapBuilder.newMapBuilder();
    CloudBlobClient client = this.getSelectedClient(account, mode);
    CloudBlobContainer blobContainer = client.getContainerReference(container);

    SocketAccess.doPrivilegedVoidException(() -> {
        if (blobContainer.exists()) {
            for (ListBlobItem blobItem : blobContainer.listBlobs(keyPath + (prefix == null ? "" : prefix))) {
                URI uri = blobItem.getUri();
                logger.trace("blob url [{}]", uri);

                // uri.getPath is of the form /container/keyPath.* and we want to strip off the /container/
                // this requires 1 + container.length() + 1, with each 1 corresponding to one of the /
                String blobPath = uri.getPath().substring(1 + container.length() + 1);

                CloudBlockBlob blob = blobContainer.getBlockBlobReference(blobPath);

                // fetch the blob attributes from Azure (getBlockBlobReference does not do this)
                // this is needed to retrieve the blob length (among other metadata) from Azure Storage
                blob.downloadAttributes();

                BlobProperties properties = blob.getProperties();
                String name = blobPath.substring(keyPath.length());
                logger.trace("blob url [{}], name [{}], size [{}]", uri, name, properties.getLength());
                blobsBuilder.put(name, new PlainBlobMetaData(name, properties.getLength()));
            }
        }
    });

    return blobsBuilder.immutableMap();
}
 
开发者ID:justor,项目名称:elasticsearch_my,代码行数:38,代码来源:AzureStorageServiceImpl.java


示例7: listBlobsByPrefix

import org.elasticsearch.common.blobstore.BlobMetaData; //导入依赖的package包/类
@Override
public Map<String, BlobMetaData> listBlobsByPrefix(String account, LocationMode mode, String container, String keyPath, String prefix) {
    MapBuilder<String, BlobMetaData> blobsBuilder = MapBuilder.newMapBuilder();
    blobs.forEach((String blobName, ByteArrayOutputStream bos) -> {
        final String checkBlob;
        if (keyPath != null && !keyPath.isEmpty()) {
            // strip off key path from the beginning of the blob name
            checkBlob = blobName.replace(keyPath, "");
        } else {
            checkBlob = blobName;
        }
        if (prefix == null || startsWithIgnoreCase(checkBlob, prefix)) {
            blobsBuilder.put(blobName, new PlainBlobMetaData(checkBlob, bos.size()));
        }
    });
    return blobsBuilder.immutableMap();
}
 
开发者ID:justor,项目名称:elasticsearch_my,代码行数:18,代码来源:AzureStorageServiceMock.java


示例8: findLatestFileNameGeneration

import org.elasticsearch.common.blobstore.BlobMetaData; //导入依赖的package包/类
/**
 * Finds the next available blob number
 *
 * @param blobs list of blobs in the repository
 * @return next available blob number
 */
protected long findLatestFileNameGeneration(Map<String, BlobMetaData> blobs) {
    long generation = -1;
    for (String name : blobs.keySet()) {
        if (!name.startsWith(DATA_BLOB_PREFIX)) {
            continue;
        }
        name = FileInfo.canonicalName(name);
        try {
            long currentGen = Long.parseLong(name.substring(DATA_BLOB_PREFIX.length()), Character.MAX_RADIX);
            if (currentGen > generation) {
                generation = currentGen;
            }
        } catch (NumberFormatException e) {
            logger.warn("file [{}] does not conform to the '{}' schema", name, DATA_BLOB_PREFIX);
        }
    }
    return generation;
}
 
开发者ID:baidu,项目名称:Elasticsearch,代码行数:25,代码来源:BlobStoreIndexShardRepository.java


示例9: listBlobsByPrefix

import org.elasticsearch.common.blobstore.BlobMetaData; //导入依赖的package包/类
@Override
public ImmutableMap<String, BlobMetaData> listBlobsByPrefix(String blobNamePrefix) throws IOException {
    // using MapBuilder and not ImmutableMap.Builder as it seems like File#listFiles might return duplicate files!
    MapBuilder<String, BlobMetaData> builder = MapBuilder.newMapBuilder();

    blobNamePrefix = blobNamePrefix == null ? "" : blobNamePrefix;
    try (DirectoryStream<Path> stream = Files.newDirectoryStream(path, blobNamePrefix + "*")) {
        for (Path file : stream) {
            final BasicFileAttributes attrs = Files.readAttributes(file, BasicFileAttributes.class);
            if (attrs.isRegularFile()) {
                builder.put(file.getFileName().toString(), new PlainBlobMetaData(file.getFileName().toString(), attrs.size()));
            }
        }
    }
    return builder.immutableMap();
}
 
开发者ID:baidu,项目名称:Elasticsearch,代码行数:17,代码来源:FsBlobContainer.java


示例10: listBlobs

import org.elasticsearch.common.blobstore.BlobMetaData; //导入依赖的package包/类
@Override
public Map<String, BlobMetaData> listBlobs() throws IOException {
    FileStatus[] files = SecurityUtils.execute(blobStore.fileSystemFactory(), new FsCallback<FileStatus[]>() {
        @Override
        public FileStatus[] doInHdfs(FileSystem fs) throws IOException {
            return fs.listStatus(path);
        }
    });
    if (files == null || files.length == 0) {
        return Collections.emptyMap();
    }
    Map<String, BlobMetaData> map = new LinkedHashMap<String, BlobMetaData>();
    for (FileStatus file : files) {
        map.put(file.getPath().getName(), new PlainBlobMetaData(file.getPath().getName(), file.getLen()));
    }
    return Collections.unmodifiableMap(map);
}
 
开发者ID:xushjie1987,项目名称:es-hadoop-v2.2.0,代码行数:18,代码来源:HdfsBlobContainer.java


示例11: listBlobsByPrefix

import org.elasticsearch.common.blobstore.BlobMetaData; //导入依赖的package包/类
@Override
public ImmutableMap<String, BlobMetaData> listBlobsByPrefix(@Nullable String blobNamePrefix) throws IOException {
    ImmutableMap.Builder<String, BlobMetaData> blobsBuilder = ImmutableMap.builder();

    List<GridFSDBFile> files;
    if (blobNamePrefix != null) {
        files = blobStore.gridFS().find(new BasicDBObject("filename", "/^" + buildKey(blobNamePrefix) + "/"));
    } else {
        files = blobStore.gridFS().find(new BasicDBObject("filename", "/^" + keyPath + "/"));
    }
     if (files != null && !files.isEmpty()) {
        for (GridFSDBFile file : files) {
            String name = file.getFilename().substring(keyPath.length());
            blobsBuilder.put(name, new PlainBlobMetaData(name, file.getLength()));
        }
    }
    return blobsBuilder.build();
}
 
开发者ID:kzwang,项目名称:elasticsearch-repository-gridfs,代码行数:19,代码来源:AbstractGridFsBlobContainer.java


示例12: listBlobsByPrefix

import org.elasticsearch.common.blobstore.BlobMetaData; //导入依赖的package包/类
/**
 * Get the blobs matching a given prefix
 * @param blobNamePrefix The prefix to look for blobs with
 * @return blobs metadata
 */
@Override
public ImmutableMap<String, BlobMetaData> listBlobsByPrefix(@Nullable final String blobNamePrefix) {
    return SwiftPerms.exec(new PrivilegedAction<ImmutableMap<String, BlobMetaData>>() {
        @Override
        public ImmutableMap<String, BlobMetaData> run() {
            ImmutableMap.Builder<String, BlobMetaData> blobsBuilder = ImmutableMap.builder();
            Collection<DirectoryOrObject> files;
            if (blobNamePrefix != null) {
                files = blobStore.swift().listDirectory(new Directory(buildKey(blobNamePrefix), '/'));
            } else {
                files = blobStore.swift().listDirectory(new Directory(keyPath, '/'));
            }
            if (files != null && !files.isEmpty()) {
                for (DirectoryOrObject object : files) {
                    if (object.isObject()) {
                        String name = object.getName().substring(keyPath.length());
                        blobsBuilder.put(name, new PlainBlobMetaData(name, object.getAsObject().getContentLength()));
                    }
                }
            }
            return blobsBuilder.build();
        }
    });
}
 
开发者ID:wikimedia,项目名称:search-repository-swift,代码行数:30,代码来源:SwiftBlobContainer.java


示例13: listBlobsByPrefix

import org.elasticsearch.common.blobstore.BlobMetaData; //导入依赖的package包/类
@Override
public Map<String, BlobMetaData> listBlobsByPrefix(
    String blobNamePrefix) throws IOException {
    try {
        final Vector<LsEntry> entries = blobStore.getClient().ls(path());
        if (entries.isEmpty()) {
            return new HashMap<>();
        }

        final String namePrefix = blobNamePrefix == null ? ""
            : blobNamePrefix;
        final MapBuilder<String, BlobMetaData> builder = MapBuilder
            .newMapBuilder();
        for (final LsEntry entry : entries) {
            if (entry.getAttrs().isReg()
                && entry.getFilename().startsWith(namePrefix)) {
                builder.put(entry.getFilename(), new PlainBlobMetaData(
                    entry.getFilename(), entry.getAttrs().getSize()));
            }
        }
        return builder.immutableMap();
    } catch (Exception e) {
        throw new IOException("Failed to load files in " + path().buildAsString("/"), e);
    }
}
 
开发者ID:codelibs,项目名称:elasticsearch-repository-ssh,代码行数:26,代码来源:SshBlobContainer.java


示例14: listBlobs

import org.elasticsearch.common.blobstore.BlobMetaData; //导入依赖的package包/类
@Override
public ImmutableMap<String, BlobMetaData> listBlobs(URL url) throws IOException {
    HttpURLConnection con = (HttpURLConnection) url.openConnection();
    try {
        if (basicAuth != null) {
            con.setRequestProperty("Authorization", basicAuth);
        }
        setPropFindMethod(con);

        int statusCode = con.getResponseCode();
        String responseMessage = con.getResponseMessage();
        if (statusCode >= SC_OK && statusCode < SC_MULTIPLE_CHOICES) {
            return PropFindResponseParser.parse(con.getInputStream());
        }
        if (statusCode == SC_NOT_FOUND) {
            return emptyMap;
        }
        throw new IOException("status code: " + statusCode + " message: " + responseMessage);
    } finally {
        con.disconnect();
    }
}
 
开发者ID:mitallast,项目名称:elasticsearch-webdav-plugin,代码行数:23,代码来源:UrlWebdavClient.java


示例15: listBlobs

import org.elasticsearch.common.blobstore.BlobMetaData; //导入依赖的package包/类
@Override
public ImmutableMap<String, BlobMetaData> listBlobs(URL url) throws IOException {
    ImmutableMap.Builder<String, BlobMetaData> builder = ImmutableMap.builder();
    DavEntry davEntry = getEntry(url);
    if (davEntry == null) {
        return builder.build();
    }
    for (Map.Entry<String, DavEntry> resourceEntry : davEntry.children.entrySet()) {
        BlobMetaData blobMetaData = new MockBlobMetaData(
            resourceEntry.getKey(),
            resourceEntry.getValue().outputStream != null
                ? resourceEntry.getValue().outputStream.size()
                : 0);
        builder.put(resourceEntry.getKey(), blobMetaData);
    }
    return builder.build();
}
 
开发者ID:mitallast,项目名称:elasticsearch-webdav-plugin,代码行数:18,代码来源:TestWebdavService.java


示例16: listBlobsByPrefix

import org.elasticsearch.common.blobstore.BlobMetaData; //导入依赖的package包/类
@Override
public Map<String, BlobMetaData> listBlobsByPrefix(@Nullable String prefix) throws IOException {
    logger.trace("listBlobsByPrefix({})", prefix);

    try {
        return blobStore.listBlobsByPrefix(blobStore.container(), keyPath, prefix);
    } catch (URISyntaxException | StorageException e) {
        logger.warn("can not access [{}] in container {{}}: {}", prefix, blobStore.container(), e.getMessage());
        throw new IOException(e);
    }
}
 
开发者ID:justor,项目名称:elasticsearch_my,代码行数:12,代码来源:AzureBlobContainer.java


示例17: listBlobsByPrefix

import org.elasticsearch.common.blobstore.BlobMetaData; //导入依赖的package包/类
@Override
public Map<String, BlobMetaData> listBlobsByPrefix(@Nullable String blobNamePrefix) throws IOException {
    return AccessController.doPrivileged((PrivilegedAction<Map<String, BlobMetaData>>) () -> {
        MapBuilder<String, BlobMetaData> blobsBuilder = MapBuilder.newMapBuilder();
        AmazonS3 client = blobStore.client();
        SocketAccess.doPrivilegedVoid(() -> {
            ObjectListing prevListing = null;
            while (true) {
                ObjectListing list;
                if (prevListing != null) {
                    list = client.listNextBatchOfObjects(prevListing);
                } else {
                    if (blobNamePrefix != null) {
                        list = client.listObjects(blobStore.bucket(), buildKey(blobNamePrefix));
                    } else {
                        list = client.listObjects(blobStore.bucket(), keyPath);
                    }
                }
                for (S3ObjectSummary summary : list.getObjectSummaries()) {
                    String name = summary.getKey().substring(keyPath.length());
                    blobsBuilder.put(name, new PlainBlobMetaData(name, summary.getSize()));
                }
                if (list.isTruncated()) {
                    prevListing = list;
                } else {
                    break;
                }
            }
        });
        return blobsBuilder.immutableMap();
    });
}
 
开发者ID:justor,项目名称:elasticsearch_my,代码行数:33,代码来源:S3BlobContainer.java


示例18: listBlobsByPrefix

import org.elasticsearch.common.blobstore.BlobMetaData; //导入依赖的package包/类
@Override
public Map<String, BlobMetaData> listBlobsByPrefix(@Nullable final String prefix) throws IOException {
    FileStatus[] files = store.execute(fileContext -> (fileContext.util().listStatus(path,
        path -> prefix == null || path.getName().startsWith(prefix))));
    Map<String, BlobMetaData> map = new LinkedHashMap<String, BlobMetaData>();
    for (FileStatus file : files) {
        map.put(file.getPath().getName(), new PlainBlobMetaData(file.getPath().getName(), file.getLen()));
    }
    return Collections.unmodifiableMap(map);
}
 
开发者ID:justor,项目名称:elasticsearch_my,代码行数:11,代码来源:HdfsBlobContainer.java


示例19: deleteBlobsByPrefix

import org.elasticsearch.common.blobstore.BlobMetaData; //导入依赖的package包/类
@Override
public void deleteBlobsByPrefix(final String blobNamePrefix) throws IOException {
    Map<String, BlobMetaData> blobs = listBlobsByPrefix(blobNamePrefix);
    for (BlobMetaData blob : blobs.values()) {
        deleteBlob(blob.name());
    }
}
 
开发者ID:baidu,项目名称:Elasticsearch,代码行数:8,代码来源:AbstractBlobContainer.java


示例20: listBlobs

import org.elasticsearch.common.blobstore.BlobMetaData; //导入依赖的package包/类
/**
 * This operation is not supported by URLBlobContainer
 */
@Override
public Map<String, BlobMetaData> listBlobs() throws IOException {
    throw new UnsupportedOperationException("URL repository doesn't support this operation");
}
 
开发者ID:justor,项目名称:elasticsearch_my,代码行数:8,代码来源:URLBlobContainer.java



注:本文中的org.elasticsearch.common.blobstore.BlobMetaData类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。


鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
上一篇:
Java ViewStubCompat类代码示例发布时间:2022-05-23
下一篇:
Java CommandSetDefaultSpawnpoint类代码示例发布时间:2022-05-23
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap