本文整理汇总了Java中org.embulk.spi.FileInput类的典型用法代码示例。如果您正苦于以下问题：Java FileInput类的具体用法？Java FileInput怎么用？Java FileInput使用的例子？那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。
FileInput类属于org.embulk.spi包，在下文中一共展示了FileInput类的20个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于我们的系统推荐出更棒的Java代码示例。
示例1: testOpenForGeneratedArchives
import org.embulk.spi.FileInput; //导入依赖的package包/类
@Test
public void testOpenForGeneratedArchives() throws Exception
{
String[] testFormats = new String[]{
ArchiveStreamFactory.AR,
// ArchiveStreamFactory.ARJ, // ArchiveException: Archiver: arj not found.
ArchiveStreamFactory.CPIO,
// ArchiveStreamFactory.DUMP, // ArchiveException: Archiver: dump not found.
ArchiveStreamFactory.JAR,
// ArchiveStreamFactory.SEVEN_Z, // StreamingNotSupportedException: The 7z doesn't support streaming.
ArchiveStreamFactory.TAR,
ArchiveStreamFactory.ZIP,
};
for (String format : testFormats) {
TaskSource mockTaskSource = new MockTaskSource(format);
FileInput mockInput = new MockFileInput(
getInputStreamAsBuffer(
getArchiveInputStream(format, "sample_1.csv", "sample_2.csv")));
CommonsCompressDecoderPlugin plugin = new CommonsCompressDecoderPlugin();
FileInput archiveFileInput = plugin.open(mockTaskSource, mockInput);
verifyContents(archiveFileInput, "1,foo", "2,bar");
}
}
开发者ID:hata，项目名称:embulk-decoder-commons-compress，代码行数:25，代码来源:TestCommonsCompressDecoderPlugin.java
示例2: open
import org.embulk.spi.FileInput; //导入依赖的package包/类
@Override
public FileInput open(TaskSource taskSource, FileInput fileInput) {
final PluginTask task = taskSource.loadTask(PluginTask.class);
final FileInputInputStream files = new FileInputInputStream(fileInput);
return new InputStreamFileInput(
task.getBufferAllocator(),
new InputStreamFileInput.Provider() {
public InputStream openNext() throws IOException {
if (!files.nextFile()) {
return null;
}
LOGGER.debug("generate LZ4FrameInputStream for {}", files.toString());
return new LZ4FrameInputStream(files);
}
public void close() throws IOException {
files.close();
}
}
);
}
开发者ID:yuuzi41，项目名称:embulk-decoder-lz4，代码行数:24，代码来源:Lz4DecoderPlugin.java
示例3: open
import org.embulk.spi.FileInput; //导入依赖的package包/类
@Override
public FileInput open(TaskSource taskSource, FileInput input)
{
PluginTask task = taskSource.loadTask(PluginTask.class);
return new CommonsCompressFileInput(
task.getBufferAllocator(),
new CommonsCompressProvider(task, new FileInputInputStream(input) {
// NOTE: This is workaround code to avoid hanging issue.
// This issue will be fixed after merging #112.
// https://github.com/embulk/embulk/pull/112
@Override
public long skip(long len) {
long skipped = super.skip(len);
return skipped > 0 ? skipped : 0;
}
}));
}
开发者ID:hata，项目名称:embulk-decoder-commons-compress，代码行数:18，代码来源:CommonsCompressDecoderPlugin.java
示例4: testOpenForNoFile
import org.embulk.spi.FileInput; //导入依赖的package包/类
@Test
public void testOpenForNoFile(@Mocked final FileInput input) throws Exception
{
new NonStrictExpectations() {{
taskSource.loadTask(CommonsCompressDecoderPlugin.PluginTask.class); result = task;
task.getFormat(); result = "tar";
input.nextFile(); result = true; result = false;
input.poll(); result = getResourceAsBuffer("sample_0.tar");
task.getBufferAllocator(); result = newBufferAllocator();
}};
CommonsCompressDecoderPlugin plugin = new CommonsCompressDecoderPlugin();
FileInput archiveFileInput = plugin.open(taskSource, input);
Assert.assertFalse("Verify there is no file.", archiveFileInput.nextFile());
archiveFileInput.close();
new Verifications() {{
input.nextFile(); times = 2;
input.close(); times = 1;
}};
}
开发者ID:hata，项目名称:embulk-decoder-commons-compress，代码行数:23，代码来源:TestCommonsCompressDecoderPlugin.java
示例5: testOpenForOneFile
import org.embulk.spi.FileInput; //导入依赖的package包/类
@Test
public void testOpenForOneFile(@Mocked final FileInput input) throws Exception
{
new NonStrictExpectations() {{
taskSource.loadTask(CommonsCompressDecoderPlugin.PluginTask.class); result = task;
task.getFormat(); result = "tar";
input.nextFile(); result = true; result = false;
input.poll(); result = getResourceAsBuffer("sample_1.tar");
task.getBufferAllocator(); result = newBufferAllocator();
}};
CommonsCompressDecoderPlugin plugin = new CommonsCompressDecoderPlugin();
FileInput archiveFileInput = plugin.open(taskSource, input);
verifyContents(archiveFileInput, "1,foo");
new Verifications() {{
input.nextFile(); times = 2;
input.close(); times = 1;
}};
}
开发者ID:hata，项目名称:embulk-decoder-commons-compress，代码行数:22，代码来源:TestCommonsCompressDecoderPlugin.java
示例6: testOpenForTwoFiles
import org.embulk.spi.FileInput; //导入依赖的package包/类
@Test
public void testOpenForTwoFiles(@Mocked final FileInput input) throws Exception
{
new NonStrictExpectations() {{
taskSource.loadTask(CommonsCompressDecoderPlugin.PluginTask.class); result = task;
task.getFormat(); result = "tar";
input.nextFile(); result = true; result = false;
input.poll(); result = getResourceAsBuffer("samples.tar");
task.getBufferAllocator(); result = newBufferAllocator();
}};
CommonsCompressDecoderPlugin plugin = new CommonsCompressDecoderPlugin();
FileInput archiveFileInput = plugin.open(taskSource, input);
verifyContents(archiveFileInput, "1,foo", "2,bar");
new Verifications() {{
input.nextFile(); times = 2;
input.close(); times = 1;
}};
}
开发者ID:hata，项目名称:embulk-decoder-commons-compress，代码行数:22，代码来源:TestCommonsCompressDecoderPlugin.java
示例7: testOpenForFourFiles
import org.embulk.spi.FileInput; //导入依赖的package包/类
@Test
public void testOpenForFourFiles(@Mocked final FileInput input) throws Exception
{
new NonStrictExpectations() {{
taskSource.loadTask(CommonsCompressDecoderPlugin.PluginTask.class); result = task;
task.getFormat(); result = "zip";
input.nextFile(); result = true; result = true; result = false; // two files.
input.poll(); result = getResourceAsBuffer("samples.zip"); result = getResourceAsBuffer("samples.zip");
task.getBufferAllocator(); result = newBufferAllocator();
}};
CommonsCompressDecoderPlugin plugin = new CommonsCompressDecoderPlugin();
FileInput archiveFileInput = plugin.open(taskSource, input);
verifyContents(archiveFileInput, "1,foo", "2,bar", "1,foo", "2,bar");
new Verifications() {{
input.nextFile(); times = 3;
input.close(); times = 1;
}};
}
开发者ID:hata，项目名称:embulk-decoder-commons-compress，代码行数:22，代码来源:TestCommonsCompressDecoderPlugin.java
示例8: testOpenArchiveFormatAutoDetect
import org.embulk.spi.FileInput; //导入依赖的package包/类
@Test
public void testOpenArchiveFormatAutoDetect(@Mocked final FileInput input) throws Exception
{
new NonStrictExpectations() {{
taskSource.loadTask(CommonsCompressDecoderPlugin.PluginTask.class); result = task;
task.getFormat(); result = "";
input.nextFile(); result = true; result = false;
input.poll(); result = getResourceAsBuffer("sample_1.tar");
task.getBufferAllocator(); result = newBufferAllocator();
}};
CommonsCompressDecoderPlugin plugin = new CommonsCompressDecoderPlugin();
FileInput archiveFileInput = plugin.open(taskSource, input);
verifyContents(archiveFileInput, "1,foo");
new Verifications() {{
input.nextFile(); times = 2;
input.close(); times = 1;
}};
}
开发者ID:hata，项目名称:embulk-decoder-commons-compress，代码行数:22，代码来源:TestCommonsCompressDecoderPlugin.java
示例9: testOpenForGeneratedCompression
import org.embulk.spi.FileInput; //导入依赖的package包/类
@Test
public void testOpenForGeneratedCompression() throws Exception
{
String[] testFormats = new String[]{
CompressorStreamFactory.BZIP2,
CompressorStreamFactory.DEFLATE,
CompressorStreamFactory.GZIP,
// CompressorStreamFactory.LZMA, // CompressorException: Compressor: lzma not found.
// CompressorStreamFactory.PACK200, // Failed to generate compressed file.
// CompressorStreamFactory.SNAPPY_FRAMED, // CompressorException: Compressor: snappy-framed not found.
// CompressorStreamFactory.SNAPPY_RAW, // CompressorException: Compressor: snappy-raw not found.
// CompressorStreamFactory.XZ, // ClassNotFoundException: org.tukaani.xz.FilterOptions
// CompressorStreamFactory.Z, // CompressorException: Compressor: z not found.
};
for (String format : testFormats) {
TaskSource mockTaskSource = new MockTaskSource(format);
FileInput mockInput = new MockFileInput(
getInputStreamAsBuffer(
getCompressorInputStream(format, "sample_1.csv")));
CommonsCompressDecoderPlugin plugin = new CommonsCompressDecoderPlugin();
FileInput archiveFileInput = plugin.open(mockTaskSource, mockInput);
verifyContents(archiveFileInput, "1,foo");
}
}
开发者ID:hata，项目名称:embulk-decoder-commons-compress，代码行数:26，代码来源:TestCommonsCompressDecoderPlugin.java
示例10: testOpenForTGZFormat
import org.embulk.spi.FileInput; //导入依赖的package包/类
@Test
public void testOpenForTGZFormat(@Mocked final FileInput input) throws Exception
{
new NonStrictExpectations() {{
taskSource.loadTask(CommonsCompressDecoderPlugin.PluginTask.class); result = task;
task.getFormat(); result = "tgz";
input.nextFile(); result = true; result = false;
input.poll(); result = getResourceAsBuffer("samples.tgz");
task.getBufferAllocator(); result = newBufferAllocator();
}};
CommonsCompressDecoderPlugin plugin = new CommonsCompressDecoderPlugin();
FileInput archiveFileInput = plugin.open(taskSource, input);
verifyContents(archiveFileInput, "1,foo", "2,bar");
new Verifications() {{
input.nextFile(); times = 2;
input.close(); times = 1;
}};
}
开发者ID:hata，项目名称:embulk-decoder-commons-compress，代码行数:22，代码来源:TestCommonsCompressDecoderPlugin.java
示例11: testOpenForTarGZFormat
import org.embulk.spi.FileInput; //导入依赖的package包/类
@Test
public void testOpenForTarGZFormat(@Mocked final FileInput input) throws Exception
{
new NonStrictExpectations() {{
taskSource.loadTask(CommonsCompressDecoderPlugin.PluginTask.class); result = task;
task.getFormat(); result = "tar.gz";
input.nextFile(); result = true; result = false;
input.poll(); result = getResourceAsBuffer("samples.tar.gz");
task.getBufferAllocator(); result = newBufferAllocator();
}};
CommonsCompressDecoderPlugin plugin = new CommonsCompressDecoderPlugin();
FileInput archiveFileInput = plugin.open(taskSource, input);
verifyContents(archiveFileInput, "1,foo", "2,bar");
new Verifications() {{
input.nextFile(); times = 2;
input.close(); times = 1;
}};
}
开发者ID:hata，项目名称:embulk-decoder-commons-compress，代码行数:22，代码来源:TestCommonsCompressDecoderPlugin.java
示例12: testOpenForTarBZ2Format
import org.embulk.spi.FileInput; //导入依赖的package包/类
@Test
public void testOpenForTarBZ2Format(@Mocked final FileInput input) throws Exception
{
new NonStrictExpectations() {{
taskSource.loadTask(CommonsCompressDecoderPlugin.PluginTask.class); result = task;
task.getFormat(); result = "tar.bz2";
input.nextFile(); result = true; result = false;
input.poll(); result = getResourceAsBuffer("samples.tar.bz2");
task.getBufferAllocator(); result = newBufferAllocator();
}};
CommonsCompressDecoderPlugin plugin = new CommonsCompressDecoderPlugin();
FileInput archiveFileInput = plugin.open(taskSource, input);
verifyContents(archiveFileInput, "1,foo", "2,bar");
new Verifications() {{
input.nextFile(); times = 2;
input.close(); times = 1;
}};
}
开发者ID:hata，项目名称:embulk-decoder-commons-compress，代码行数:22，代码来源:TestCommonsCompressDecoderPlugin.java
示例13: testOpenForTarZFormat
import org.embulk.spi.FileInput; //导入依赖的package包/类
@Test
public void testOpenForTarZFormat(@Mocked final FileInput input) throws Exception
{
new NonStrictExpectations() {{
taskSource.loadTask(CommonsCompressDecoderPlugin.PluginTask.class); result = task;
task.getFormat(); result = "tar.Z";
input.nextFile(); result = true; result = false;
input.poll(); result = getResourceAsBuffer("samples.tar.Z");
task.getBufferAllocator(); result = newBufferAllocator();
}};
CommonsCompressDecoderPlugin plugin = new CommonsCompressDecoderPlugin();
FileInput archiveFileInput = plugin.open(taskSource, input);
verifyContents(archiveFileInput, "1,foo", "2,bar");
new Verifications() {{
input.nextFile(); times = 2;
input.close(); times = 1;
}};
}
开发者ID:hata，项目名称:embulk-decoder-commons-compress，代码行数:22，代码来源:TestCommonsCompressDecoderPlugin.java
示例14: run
import org.embulk.spi.FileInput; //导入依赖的package包/类
@Override
public void run(TaskSource taskSource, Schema schema,
FileInput input, PageOutput output)
{
PluginTask task = taskSource.loadTask(PluginTask.class);
LineDecoder lineDecoder = new LineDecoder(input,task);
PageBuilder pageBuilder = new PageBuilder(Exec.getBufferAllocator(), schema, output);
String line = null;
final String columnName = task.getColumnName();
while( input.nextFile() ){
while(true){
line = lineDecoder.poll();
if( line == null ){
break;
}
pageBuilder.setString(0, line);
pageBuilder.addRecord();
}
}
pageBuilder.finish();
}
开发者ID:sonots，项目名称:embulk-parser-none，代码行数:25，代码来源:NoneParserPlugin.java
示例15: transaction
import org.embulk.spi.FileInput; //导入依赖的package包/类
private void transaction(ConfigSource config, final FileInput input)
{
plugin.transaction(config, new ParserPlugin.Control()
{
@Override
public void run(TaskSource taskSource, Schema schema)
{
plugin.run(taskSource, schema, input, output);
}
});
}
开发者ID:joker1007，项目名称:embulk-parser-avro，代码行数:12，代码来源:TestAvroParserPlugin.java
示例16: fileInput
import org.embulk.spi.FileInput; //导入依赖的package包/类
private FileInput fileInput(String... lines)
throws Exception
{
StringBuilder sb = new StringBuilder();
for (String line : lines) {
sb.append(line).append("\n");
}
ByteArrayInputStream in = new ByteArrayInputStream(sb.toString().getBytes(StandardCharsets.UTF_8));
return new InputStreamFileInput(runtime.getBufferAllocator(), provider(in));
}
开发者ID:hiroyuki-sato，项目名称:embulk-parser-jsonpath，代码行数:12，代码来源:TestJsonpathParserPlugin.java
示例17: testOpen
import org.embulk.spi.FileInput; //导入依赖的package包/类
@Test
public void testOpen(@Mocked final FileInput input)
{
new NonStrictExpectations() {{
taskSource.loadTask(CommonsCompressDecoderPlugin.PluginTask.class); result = task;
}};
CommonsCompressDecoderPlugin plugin = new CommonsCompressDecoderPlugin();
Assert.assertNotNull("Verify a value is returned.", plugin.open(taskSource, input));
}
开发者ID:hata，项目名称:embulk-decoder-commons-compress，代码行数:12，代码来源:TestCommonsCompressDecoderPlugin.java
示例18: testOpenAutoDetectFailed
import org.embulk.spi.FileInput; //导入依赖的package包/类
@Test(expected=RuntimeException.class)
public void testOpenAutoDetectFailed(@Mocked final FileInput input) throws Exception
{
new NonStrictExpectations() {{
taskSource.loadTask(CommonsCompressDecoderPlugin.PluginTask.class); result = task;
task.getFormat(); result = "";
input.nextFile(); result = true; result = false;
input.poll(); result = getResourceAsBuffer("sample_1.csv"); // This is not an archive.
task.getBufferAllocator(); result = newBufferAllocator();
}};
CommonsCompressDecoderPlugin plugin = new CommonsCompressDecoderPlugin();
FileInput archiveFileInput = plugin.open(taskSource, input);
archiveFileInput.nextFile();
}
开发者ID:hata，项目名称:embulk-decoder-commons-compress，代码行数:16，代码来源:TestCommonsCompressDecoderPlugin.java
示例19: testOpenExplicitConfigFailed
import org.embulk.spi.FileInput; //导入依赖的package包/类
@Test(expected=RuntimeException.class)
public void testOpenExplicitConfigFailed(@Mocked final FileInput input) throws Exception
{
new NonStrictExpectations() {{
taskSource.loadTask(CommonsCompressDecoderPlugin.PluginTask.class); result = task;
task.getFormat(); result = "tar";
input.nextFile(); result = true; result = false;
input.poll(); result = getResourceAsBuffer("samples.zip"); // This is not tar file.
task.getBufferAllocator(); result = newBufferAllocator();
}};
CommonsCompressDecoderPlugin plugin = new CommonsCompressDecoderPlugin();
FileInput archiveFileInput = plugin.open(taskSource, input);
archiveFileInput.nextFile();
}
开发者ID:hata，项目名称:embulk-decoder-commons-compress，代码行数:16，代码来源:TestCommonsCompressDecoderPlugin.java
示例20: readFileInput
import org.embulk.spi.FileInput; //导入依赖的package包/类
private String readFileInput(FileInput input) throws IOException {
ByteArrayOutputStream bout = new ByteArrayOutputStream();
Buffer buffer = input.poll();
while (buffer != null) {
bout.write(buffer.array(), buffer.offset(), buffer.limit());
buffer = input.poll();
}
return bout.toString().trim();
}
开发者ID:hata，项目名称:embulk-decoder-commons-compress，代码行数:10，代码来源:TestCommonsCompressDecoderPlugin.java
注：本文中的org.embulk.spi.FileInput类示例整理自Github/MSDocs等源码及文档管理平台，相关代码片段筛选自各路编程大神贡献的开源项目，源码版权归原作者所有，传播和使用请参考对应项目的License；未经允许，请勿转载。 |
客服电话
APP下载
官方微信
请发表评论