本文整理汇总了Java中org.iq80.snappy.SnappyInputStream类的典型用法代码示例。如果您正苦于以下问题：Java SnappyInputStream类的具体用法？Java SnappyInputStream怎么用？Java SnappyInputStream使用的例子？那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。
SnappyInputStream类属于org.iq80.snappy包，在下文中一共展示了SnappyInputStream类的15个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于我们的系统推荐出更棒的Java代码示例。
示例1: allFloatParameterCombinationsKnockout
import org.iq80.snappy.SnappyInputStream; //导入依赖的package包/类
@Test
public void allFloatParameterCombinationsKnockout() throws IOException {
final List<float[]> fullData = loadFullFloatData();
try (BufferedWriter noSplitWriter = openWriter("floats-knockout-nosplit");
BufferedWriter splitWriter = openWriter("floats-knockout")) {
for (float knockout : new float[] { 0f, 0.1f, 0.5f, 0.75f }) {
final Random r = new Random(1337);
final List<float[]> check = fullData.stream().map(xs -> {
final float[] ys = xs.clone();
for (int i = 0; i < ys.length; i++) {
if (r.nextDouble() < knockout) {
ys[i] = Float.NaN;
}
}
return ys;
}).collect(Collectors.toList());
allFloatParameterCombinationsWork(check, "Snappy\t" + knockout, SnappyOutputStream::new, SnappyInputStream::new, noSplitWriter, splitWriter);
}
}
}
开发者ID:batterseapower，项目名称:timeseries-compression，代码行数:23，代码来源:ConditionerParameterSearchTest.java
示例2: allDoubleParameterCombinationsKnockout
import org.iq80.snappy.SnappyInputStream; //导入依赖的package包/类
@Test
public void allDoubleParameterCombinationsKnockout() throws IOException {
final List<double[]> fullData = loadFullDoubleData();
try (BufferedWriter noSplitWriter = openWriter("doubles-knockout-nosplit");
BufferedWriter splitWriter = openWriter("doubles-knockout")) {
for (float knockout : new float[] { 0f, 0.1f, 0.5f, 0.75f }) {
final Random r = new Random(1337);
final List<double[]> check = fullData.stream().map(xs -> {
final double[] ys = xs.clone();
for (int i = 0; i < ys.length; i++) {
if (r.nextDouble() < knockout) {
ys[i] = Double.NaN;
}
}
return ys;
}).collect(Collectors.toList());
allDoubleParameterCombinationsWork(check, "Snappy\t" + knockout, SnappyOutputStream::new, SnappyInputStream::new, noSplitWriter, splitWriter);
}
}
}
开发者ID:batterseapower，项目名称:timeseries-compression，代码行数:23，代码来源:ConditionerParameterSearchTest.java
示例3: init
import org.iq80.snappy.SnappyInputStream; //导入依赖的package包/类
@Override
public void init(File file) {
// currentFile = file;
try {
is = new DataInputStream(new SnappyInputStream(
new BufferedInputStream(new FileInputStream(file),
65536)));
signature[0] = triple[0];
signature[1] = triple[1];
signature[2] = triple[2];
signature[3] = step;
signature[4] = count;
} catch (Exception e) {
log.error("Error", e);
}
}
开发者ID:jrbn，项目名称:dynamite，代码行数:17，代码来源:TripleFileStorage.java
示例4: benchmark
import org.iq80.snappy.SnappyInputStream; //导入依赖的package包/类
@Test
public void benchmark() throws IOException {
/*
Snappy compress: 0.156672s
Compressed to 10876881 bytes
Uncompresses to 12908620 bytes
Snappy decompress: 0.162463s
Deflate Fastest compress: 6.017877s
Compressed to 7770401 bytes
Uncompresses to 12908620 bytes
Deflate Fastest decompress: 4.260146s
Deflate Slowest compress: 7.06043s
Compressed to 7492001 bytes
Uncompresses to 12908620 bytes
Deflate Slowest decompress: 3.968221s
Deflate Normal compress: 5.201332s
Compressed to 7515522 bytes
Uncompresses to 12908620 bytes
Deflate Normal decompress: 4.056792s
*/
benchmark("None", x -> x, x -> x);
benchmark("None", x -> x, x -> x);
benchmark("Snappy", SnappyOutputStream::new, SnappyInputStream::new);
benchmark("GZip", GZIPOutputStream::new, GZIPInputStream::new);
benchmark("Deflate Fastest", os -> new DeflaterOutputStream(os, new Deflater(Deflater.BEST_SPEED)), InflaterInputStream::new);
benchmark("Deflate Normal", os -> new DeflaterOutputStream(os, new Deflater(Deflater.DEFAULT_COMPRESSION)), InflaterInputStream::new);
benchmark("Deflate Slowest", os -> new DeflaterOutputStream(os, new Deflater(Deflater.BEST_COMPRESSION)), InflaterInputStream::new);
benchmark("BZip2 Fastest", os -> new BZip2CompressorOutputStream(os, BZip2CompressorOutputStream.MIN_BLOCKSIZE), BZip2CompressorInputStream::new);
benchmark("BZip2 Slowest", os -> new BZip2CompressorOutputStream(os, BZip2CompressorOutputStream.MAX_BLOCKSIZE), BZip2CompressorInputStream::new);
benchmark("XZ Fastest", os -> new XZCompressorOutputStream(os, LZMA2Options.PRESET_MIN), XZCompressorInputStream::new);
benchmark("XZ Normal", os -> new XZCompressorOutputStream(os, LZMA2Options.PRESET_DEFAULT), XZCompressorInputStream::new);
benchmark("XZ Slowest", os -> new XZCompressorOutputStream(os, LZMA2Options.PRESET_MAX), XZCompressorInputStream::new);
}
开发者ID:batterseapower，项目名称:timeseries-compression，代码行数:37，代码来源:ConditionerParameterSearchTest.java
示例5: allFloatParameterCombinationsWork
import org.iq80.snappy.SnappyInputStream; //导入依赖的package包/类
@Test
public void allFloatParameterCombinationsWork() throws IOException {
try (BufferedWriter noSplitWriter = openWriter("floats-nosplit");
BufferedWriter splitWriter = openWriter("floats")) {
allFloatParameterCombinationsWork(loadFullFloatData(), "Snappy", SnappyOutputStream::new, SnappyInputStream::new, noSplitWriter, splitWriter);
allFloatParameterCombinationsWork(loadFullFloatData(), "BZ2", os -> new BZip2CompressorOutputStream(os, BZip2CompressorOutputStream.MIN_BLOCKSIZE), BZip2CompressorInputStream::new, noSplitWriter, splitWriter);
}
}
开发者ID:batterseapower，项目名称:timeseries-compression，代码行数:9，代码来源:ConditionerParameterSearchTest.java
示例6: allDoubleParameterCombinationsWork
import org.iq80.snappy.SnappyInputStream; //导入依赖的package包/类
@Test
public void allDoubleParameterCombinationsWork() throws IOException {
try (BufferedWriter noSplitWriter = openWriter("doubles-nosplit");
BufferedWriter splitWriter = openWriter("doubles")) {
allDoubleParameterCombinationsWork(loadFullDoubleData(), "Snappy", SnappyOutputStream::new, SnappyInputStream::new, noSplitWriter, splitWriter);
allDoubleParameterCombinationsWork(loadFullDoubleData(), "BZ2", os -> new BZip2CompressorOutputStream(os, BZip2CompressorOutputStream.MIN_BLOCKSIZE), BZip2CompressorInputStream::new, noSplitWriter, splitWriter);
}
}
开发者ID:batterseapower，项目名称:timeseries-compression，代码行数:9，代码来源:ConditionerParameterSearchTest.java
示例7: readListBlockFromFile
import org.iq80.snappy.SnappyInputStream; //导入依赖的package包/类
ListBlocks readListBlockFromFile(int no) throws Exception {
String dir = index.cacheDir;
String name = "lb-" + no;
if (!listBlockNameSet.contains(name)) {
return null;
}
getIOLock();
File file = index.getCacheFile(name + "/index");
try {
InputStream reader = index.getFilesInterface().createInputStream(
file);
DataInputStream stream = new DataInputStream(new SnappyInputStream(
reader));
// reader);
ListBlocks block = new ListBlocks(no, name, 1);
block.readFrom(stream);
stream.close();
reader.close();
if (log.isDebugEnabled()) {
log.debug("done reading index of ListBlock " + dir + "/" + name);
}
return block;
} catch (Exception e) {
log.error("Failed reading the file: " + file, e);
throw e;
} finally {
releaseIOLock();
}
}
开发者ID:jrbn，项目名称:querypie，代码行数:30，代码来源:FirstLayer.java
示例8: openStream
import org.iq80.snappy.SnappyInputStream; //导入依赖的package包/类
@Override
public void openStream() throws IOException {
if (stream == null) {
stream = new FDataInput(new SnappyInputStream(
new BufferedInputStream(new FileInputStream(filename))));
}
}
开发者ID:jrbn，项目名称:ajira，代码行数:8，代码来源:FileMetaData.java
示例9: InputStream
import org.iq80.snappy.SnappyInputStream; //导入依赖的package包/类
public InputStream(java.io.InputStream stream) throws IOException {
myUnderlyingInputStream = new SnappyInputStream(stream);
myInputStream = new CompactDataInput(myUnderlyingInputStream);
}
开发者ID:jskierbi，项目名称:intellij-ce-playground，代码行数:5，代码来源:OutputInfoSerializer.java
示例10: snappy
import org.iq80.snappy.SnappyInputStream; //导入依赖的package包/类
@Benchmark
public int snappy() throws IOException {
return benchmark(SnappyOutputStream::new, SnappyInputStream::new);
}
开发者ID:batterseapower，项目名称:timeseries-compression，代码行数:5，代码来源:JMHCompressorTest.java
示例11: input
import org.iq80.snappy.SnappyInputStream; //导入依赖的package包/类
@Override
public InputStream input(InputStream in) throws IOException {
return new SnappyInputStream(in);
}
开发者ID:yammer，项目名称:backups，代码行数:5，代码来源:SnappyCompressionCodec.java
示例12: readBlock
import org.iq80.snappy.SnappyInputStream; //导入依赖的package包/类
int[] readBlock(String name, int blockNo, ActionContext context)
throws Exception {
getIOLock();
File f = index.getCacheFile(name + "/" + blockNo);
int[] result = null;
try {
if (log.isDebugEnabled()) {
log.debug("Reading block " + blockNo + " of " + name);
}
long time = System.currentTimeMillis();
InputStream reader = index.getFilesInterface().createInputStream(f);
DataInputStream stream = new DataInputStream(new SnappyInputStream(
reader));
// reader);
int sz = stream.readInt();
result = new int[sz];
result[0] = sz;
synchronized (tmpBuffer) {
if (tmpBuffer.length < (sz - 1) * 4) {
tmpBuffer = new byte[(sz - 1) * 4];
intBuffer = ByteBuffer.wrap(tmpBuffer).asIntBuffer();
}
stream.readFully(tmpBuffer, 0, (sz - 1) * 4);
intBuffer.rewind();
intBuffer.get(result, 1, sz - 1);
}
stream.close();
reader.close();
/*
* if (me == -1) { me = context.getNetworkLayer().getMyPartition();
* } context.incrCounter("Node " + me +
* ", time spent reading lb from disk", System.currentTimeMillis() -
* time); context.incrCounter("Node " + me +
* ", lb bytes read from disk", sz * 4);
*/
if (context != null) {
context.incrCounter("Time spent reading lb from disk",
System.currentTimeMillis() - time);
context.incrCounter("Lb bytes read from disk", sz * 4);
}
totalListBlocksSize += sz;
if (totalListBlocksSize > MAX_LISTBLOCKSSIZE_TO_KEEP_CACHE) {
// We already have the maximum number of
// ListBlocks in core, so we have
// to delete one.
removeOldestBlockInCache();
}
if (log.isDebugEnabled()) {
log.debug("After readBlock: totalListBlocksSize = "
+ totalListBlocksSize);
}
} catch (IOException e) {
log.error("Could not read block " + f.getAbsolutePath(), e);
} finally {
releaseIOLock();
}
return result;
}
开发者ID:jrbn，项目名称:querypie，代码行数:61，代码来源:FirstLayer.java
示例13: getBlock
import org.iq80.snappy.SnappyInputStream; //导入依赖的package包/类
private Block getBlock(int i, boolean removeOldest, ActionContext context) {
synchronized (blocksCache) {
Block block = blocksCache.get(myId + i);
if (block == null) {
FirstLayer.getIOLock();
File f = null;
try {
long time = System.currentTimeMillis();
f = getCacheFile("" + (i / 1000) + "/" + i);
if (log.isDebugEnabled()) {
log.debug("Reading block " + f.getPath());
}
InputStream fin = fi.createInputStream(f);
InputStream stream = new SnappyInputStream(fin);
DataInputStream din = new DataInputStream(stream);
// DataInputStream din = new DataInputStream(fin);
block = new Block();
int sz = din.readInt();
block.block = new byte[sz];
block.index = myId + i;
blocksCache.addBlock(block);
din.readFully(block.block);
din.close();
fin.close();
if (context != null) {
/*
* if (me == -1) { me =
* context.getNetworkLayer().getMyPartition(); }
* context.incrCounter( "Node " + me +
* ", time spent reading block from disk",
* System.currentTimeMillis() - time);
* context.incrCounter("Node " + me +
* ", bytes read from disk", sz+4);
*/
context.incrCounter(
"Time spent reading block from disk",
System.currentTimeMillis() - time);
context.incrCounter("Bytes read from disk", sz + 4);
}
} catch (Exception e) {
log.error("Failed reading cache file " + f.getPath(), e);
throw new Error("Failed reading cache file " + f.getPath(),
e);
} finally {
FirstLayer.releaseIOLock();
}
if (removeOldest
&& blocksCache.size() >= MAX_BUFFERS_TO_KEEP_CACHE) {
Block b = blocksCache.removeOldestBlockInCache();
if (b.marked) {
writeBlockToDisk(b);
b.marked = false;
}
}
} else {
blocksCache.moveToEnd(block);
}
return block;
}
}
开发者ID:jrbn，项目名称:querypie，代码行数:63，代码来源:Index.java
示例14: consistencyCheck
import org.iq80.snappy.SnappyInputStream; //导入依赖的package包/类
private void consistencyCheck() {
try {
stream = new FDataInput(new SnappyInputStream(
new BufferedInputStream(new FileInputStream(filename))));
long nel = nElements - 1;
long sz = size;
if (nel > 0) {
if (sz <= 0) {
log.error("inconsistency in file " + filename,
new Throwable());
throw new Error("Inconsistency");
}
}
if (nel == 0 && sz != 0) {
log.error("inconsistency in file " + filename, new Throwable());
throw new Error("Inconsistency");
}
while (nel > 0) {
// Consistency check
int length = stream.readInt();
if (length != current.length) {
current = new byte[length];
}
if (length < 0 || length > 4096) {
log.error("inconsistency in file " + filename,
new Throwable());
throw new Error("Inconsistency");
}
stream.readFully(current);
sz -= 4 + length;
nel--;
}
if (sz != 0) {
log.error("inconsistency in file " + filename, new Throwable());
throw new Error("Inconsistency");
}
stream.close();
} catch (Throwable e) {
log.error("Got exception in consistency check", e);
throw new Error("Inconsistency", e);
}
stream = null;
}
开发者ID:jrbn，项目名称:ajira，代码行数:44，代码来源:FileMetaData.java
示例15: wrapInputStream
import org.iq80.snappy.SnappyInputStream; //导入依赖的package包/类
protected InputStream wrapInputStream(InputStream underlying) throws IOException {
return new SnappyInputStream(underlying);
}
开发者ID:we7，项目名称:voldemort，代码行数:4，代码来源:SnappyCompressionStrategy.java
注：本文中的org.iq80.snappy.SnappyInputStream类示例整理自Github/MSDocs等源码及文档管理平台，相关代码片段筛选自各路编程大神贡献的开源项目，源码版权归原作者所有，传播和使用请参考对应项目的License；未经允许，请勿转载。 |
客服电话
APP下载
官方微信
请发表评论