本文整理汇总了Java中com.google.cloud.dataflow.sdk.transforms.DoFnTester类的典型用法代码示例。如果您正苦于以下问题:Java DoFnTester类的具体用法?Java DoFnTester怎么用?Java DoFnTester使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。
DoFnTester类属于com.google.cloud.dataflow.sdk.transforms包,在下文中一共展示了DoFnTester类的20个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。
示例1: testConvertJsonToReceiptDoFn
import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Test
public void testConvertJsonToReceiptDoFn() {
ConvertJsonToReceiptDoFn dofn = new ConvertJsonToReceiptDoFn();
DoFnTester<String, Receipt> fnTester = DoFnTester.of(dofn);
List<String> input = new ArrayList<>();
input.add("{\"salesDate\": \"2013-02-10T13:45:30+0900\"," +
"\"storeCode\": \"010\"," +
"\"registerID\": \"123\"," +
"\"receiptNumber\": 1," +
"\"receiptBranchNumber\": 2," +
"\"productCode\": \"301\"," +
"\"quantity\": 3," +
"\"price\": 108 }");
List<Receipt> receipts = fnTester.processBatch(input);
Receipt receipt1 = receipts.get(0);
Assert.assertThat("", receipt1.getPrice(), is(108));
Assert.assertThat("", receipt1.getQuantity(), is(3));
Assert.assertThat("", receipt1.getStoreCode(), is("010"));
Assert.assertThat("", receipt1.getProductCode(), is("301"));
Assert.assertThat("", receipt1.getRegisterID(), is("123"));
Assert.assertThat("", receipt1.getReceiptNumber(), is(1));
Assert.assertThat("", receipt1.getReceiptBranchNumber(), is(2));
}
开发者ID:topgate,项目名称:retail-demo,代码行数:24,代码来源:MainPipelineTest.java
示例2: doMutation_encodesKeysAndCounts
import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Test
public void doMutation_encodesKeysAndCounts() {
// Arrange
DoFnTester<KV<String, Integer>, Mutation> tester = DoFnTester.of(LoadBooks.ENCODE_NGRAM);
KV<String, Integer> input = KV.of("this is a test", 513);
// Act
List<Mutation> output = tester.processBatch(input);
// Assert
Put put = (Put) output.get(0);
assertThat(put.getRow()).isEqualTo("this is a test".getBytes(StandardCharsets.UTF_8));
Cell valueCell = put.get(LoadBooks.FAMILY, LoadBooks.COUNT_QUALIFIER).get(0);
byte[] valueArray = valueCell.getValueArray();
byte[] value =
Arrays.copyOfRange(
valueArray,
valueCell.getValueOffset(),
valueCell.getValueOffset() + valueCell.getValueLength());
assertThat(value).isEqualTo(new byte[] {0, 0, 2, 1});
}
开发者ID:GoogleCloudPlatform,项目名称:cloud-bigtable-examples,代码行数:22,代码来源:LoadBooksTest.java
示例3: testCSVToMapLineCombineFn
import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Test
public void testCSVToMapLineCombineFn() {
final String[] csv = new String[] {
"000,Tokyo,120",
"001,Osaka,100",
"002,Kyoto,140"
};
final List<String> csvlist = Arrays.asList(csv);
Pipeline p = TestPipeline.create();
PCollection<String> maplines = p.apply(Create.of(csvlist)).setCoder(StringUtf8Coder.of());
PCollectionView<Map<String,String>> mapview = maplines.apply(Combine.globally(new CSVToMapLineCombineFn()).asSingletonView());
final String[] dummy = new String[] {
"000",
"001",
"002"
};
List<String> dummylist = Arrays.asList(dummy);
DoFnTester<String,String> fnTester = DoFnTester.of(new AAA(mapview));
fnTester.setSideInputInGlobalWindow(mapview, csvlist);
//dummylines.apply(ParDo.of(fnTester));
List<String> results = fnTester.processBatch(dummylist);
System.out.println(results);
//p.apply()
}
开发者ID:topgate,项目名称:retail-demo,代码行数:30,代码来源:MainPipelineTest.java
示例4: testExtractWordsFn
import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
/** Example test that tests a specific DoFn. */
@Test
public void testExtractWordsFn() {
DoFnTester<String, String> extractWordsFn =
DoFnTester.of(new ExtractWordsFn());
Assert.assertThat(extractWordsFn.processBatch(" some input words "),
CoreMatchers.hasItems("some", "input", "words"));
Assert.assertThat(extractWordsFn.processBatch(" "),
CoreMatchers.<String>hasItems());
Assert.assertThat(extractWordsFn.processBatch(" some ", " input", " words"),
CoreMatchers.hasItems("some", "input", "words"));
}
开发者ID:sinmetal,项目名称:iron-hippo,代码行数:14,代码来源:WordCountTest.java
示例5: setUp
import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Before
public void setUp() throws IOException {
this.projectsApiObject = mock(Projects.class);
GCPProject.setProjectsApiStub(this.projectsApiObject);
this.getIamPolicy = mock(Projects.GetIamPolicy.class);
this.tester = DoFnTester.of(new ExtractState());
when(this.projectsApiObject.getIamPolicy(anyString(), any(GetIamPolicyRequest.class)))
.thenReturn(this.getIamPolicy);
}
开发者ID:GoogleCloudPlatform,项目名称:policyscanner,代码行数:11,代码来源:ExtractStateTest.java
示例6: setUp
import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Before
public void setUp() throws IOException {
GCPProject.setProjectsApiStub(this.projectsApiObject);
this.projectsApiObject = mock(Projects.class);
this.getIamPolicy = mock(GetIamPolicy.class);
this.tester = DoFnTester.of(new ExtractState());
when(this.projectsApiObject.getIamPolicy(anyString(), any(GetIamPolicyRequest.class)))
.thenReturn(this.getIamPolicy);
}
开发者ID:GoogleCloudPlatform,项目名称:policyscanner,代码行数:10,代码来源:ListServiceAccountsTest.java
示例7: testFilterStateNoMatchingResources
import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Test
public void testFilterStateNoMatchingResources() {
GCPProject checkedProject = getSampleProject("_checked");
GCPProject liveProject = getSampleProject("_live");
GCPResourceState checkedPolicy = getSampleGCPResourcePolicy(checkedProject, 1);
GCPResourceState livePolicy = getSampleGCPResourcePolicy(liveProject, 2);
List<KV<GCPResource, KV<StateSource, GCPResourceState>>> sideInputList =
Arrays.asList(KV.of((GCPResource) checkedProject, KV.of(StateSource.DESIRED, checkedPolicy)));
List<KV<GCPResource, KV<StateSource, GCPResourceState>>> mainInputList =
Arrays.asList(KV.of((GCPResource) liveProject, KV.of(StateSource.LIVE, livePolicy)));
PCollection<KV<GCPResource, KV<StateSource, GCPResourceState>>> collection =
pipeline.apply(Create.of(sideInputList)).setCoder(
KvCoder.of(SerializableCoder.of(GCPResource.class),
KvCoder.of(SerializableCoder.of(StateSource.class),
SerializableCoder.of(GCPResourceState.class))));
PCollectionView<Map<GCPResource, KV<StateSource, GCPResourceState>>> view =
View.<GCPResource, KV<StateSource, GCPResourceState>>asMap().apply(collection);
FilterOutMatchingState function = new FilterOutMatchingState(view);
DoFnTester<KV<GCPResource, KV<StateSource, GCPResourceState>>,
KV<GCPResource, Map<StateSource, GCPResourceState>>> tester = DoFnTester.of(function);
tester.setSideInputInGlobalWindow(view, sideInputList);
List<KV<GCPResource, Map<StateSource, GCPResourceState>>> results =
tester.processBatch(mainInputList);
assertEquals(0, results.size());
}
开发者ID:GoogleCloudPlatform,项目名称:policyscanner,代码行数:29,代码来源:FilterOutMatchingStateTest.java
示例8: testFilterStateNoMismatches
import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Test
public void testFilterStateNoMismatches() {
GCPProject project = getSampleProject("");
GCPResourceState checkedPolicy = getSampleGCPResourcePolicy(project, 1);
GCPResourceState livePolicy = checkedPolicy;
List<KV<GCPResource, KV<StateSource, GCPResourceState>>> sideInputList =
Arrays.asList(KV.of((GCPResource) project, KV.of(StateSource.DESIRED, checkedPolicy)));
List<KV<GCPResource, KV<StateSource, GCPResourceState>>> mainInputList =
Arrays.asList(KV.of((GCPResource) project, KV.of(StateSource.LIVE, livePolicy)));
PCollection<KV<GCPResource, KV<StateSource, GCPResourceState>>> collection =
pipeline.apply(Create.of(sideInputList)).setCoder(
KvCoder.of(SerializableCoder.of(GCPResource.class),
KvCoder.of(SerializableCoder.of(StateSource.class),
SerializableCoder.of(GCPResourceState.class))));
PCollectionView<Map<GCPResource, KV<StateSource, GCPResourceState>>> view =
View.<GCPResource, KV<StateSource, GCPResourceState>>asMap().apply(collection);
FilterOutMatchingState function = new FilterOutMatchingState(view);
DoFnTester<KV<GCPResource, KV<StateSource, GCPResourceState>>,
KV<GCPResource, Map<StateSource, GCPResourceState>>> tester = DoFnTester.of(function);
tester.setSideInputInGlobalWindow(view, sideInputList);
List<KV<GCPResource, Map<StateSource, GCPResourceState>>> results =
tester.processBatch(mainInputList);
assertEquals(0, results.size());
}
开发者ID:GoogleCloudPlatform,项目名称:policyscanner,代码行数:28,代码来源:FilterOutMatchingStateTest.java
示例9: testFilterStateOneMismatch
import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Test
public void testFilterStateOneMismatch() {
GCPProject project = getSampleProject("");
GCPResourceState checkedPolicy = getSampleGCPResourcePolicy(project, 1);
GCPResourceState livePolicy = getSampleGCPResourcePolicy(project, 2);
List<KV<GCPResource, KV<StateSource, GCPResourceState>>> sideInputList =
Arrays.asList(KV.of((GCPResource) project, KV.of(StateSource.DESIRED, checkedPolicy)));
List<KV<GCPResource, KV<StateSource, GCPResourceState>>> mainInputList =
Arrays.asList(KV.of((GCPResource) project, KV.of(StateSource.LIVE, livePolicy)));
PCollection<KV<GCPResource, KV<StateSource, GCPResourceState>>> collection =
pipeline.apply(Create.of(sideInputList)).setCoder(
KvCoder.of(SerializableCoder.of(GCPResource.class),
KvCoder.of(SerializableCoder.of(StateSource.class),
SerializableCoder.of(GCPResourceState.class))));
PCollectionView<Map<GCPResource, KV<StateSource, GCPResourceState>>> view =
View.<GCPResource, KV<StateSource, GCPResourceState>>asMap().apply(collection);
FilterOutMatchingState function = new FilterOutMatchingState(view);
DoFnTester<KV<GCPResource, KV<StateSource, GCPResourceState>>,
KV<GCPResource, Map<StateSource, GCPResourceState>>> tester = DoFnTester.of(function);
tester.setSideInputInGlobalWindow(view, sideInputList);
Map<StateSource, GCPResourceState> outputMap = new HashMap<>(2);
outputMap.put(StateSource.DESIRED, checkedPolicy);
outputMap.put(StateSource.LIVE, livePolicy);
List<KV<GCPResource, Map<StateSource, GCPResourceState>>> expectedOutput =
Arrays.asList(KV.of((GCPResource) project, outputMap));
List<KV<GCPResource, Map<StateSource, GCPResourceState>>> results =
tester.processBatch(mainInputList);
assertEquals(expectedOutput, results);
}
开发者ID:GoogleCloudPlatform,项目名称:policyscanner,代码行数:34,代码来源:FilterOutMatchingStateTest.java
示例10: testFilterStateNoMatchingResources
import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Test
public void testFilterStateNoMatchingResources() {
GCPProject checkedProject = getSampleProject("_checked");
GCPProject liveProject = getSampleProject("_live");
GCPResourceState checkedPolicy = getSampleGCPResourcePolicy(checkedProject, 1);
GCPResourceState livePolicy = getSampleGCPResourcePolicy(liveProject, 2);
List<KV<GCPResource, KV<StateSource, GCPResourceState>>> sideInputList =
Arrays.asList(KV.of((GCPResource) checkedProject, KV.of(StateSource.DESIRED, checkedPolicy)));
List<KV<GCPResource, KV<StateSource, GCPResourceState>>> mainInputList =
Arrays.asList(KV.of((GCPResource) liveProject, KV.of(StateSource.LIVE, livePolicy)));
PCollection<KV<GCPResource, KV<StateSource, GCPResourceState>>> collection =
pipeline.apply(Create.of(sideInputList)).setCoder(
KvCoder.of(SerializableCoder.of(GCPResource.class),
KvCoder.of(SerializableCoder.of(StateSource.class),
SerializableCoder.of(GCPResourceState.class))));
PCollectionView<Map<GCPResource, KV<StateSource, GCPResourceState>>> view =
View.<GCPResource, KV<StateSource, GCPResourceState>>asMap().apply(collection);
JoinKnownGoodAndLiveStates function = new JoinKnownGoodAndLiveStates(view);
DoFnTester<KV<GCPResource, KV<StateSource, GCPResourceState>>,
KV<GCPResource, Map<StateSource, GCPResourceState>>> tester = DoFnTester.of(function);
tester.setSideInputInGlobalWindow(view, sideInputList);
List<KV<GCPResource, Map<StateSource, GCPResourceState>>> results =
tester.processBatch(mainInputList);
assertEquals(0, results.size());
}
开发者ID:GoogleCloudPlatform,项目名称:policyscanner,代码行数:29,代码来源:JoinKnownGoodAndLiveStatesTest.java
示例11: testFilterStateNoMismatches
import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Test
public void testFilterStateNoMismatches() {
GCPProject project = getSampleProject("");
GCPResourceState checkedPolicy = getSampleGCPResourcePolicy(project, 1);
GCPResourceState livePolicy = checkedPolicy;
List<KV<GCPResource, KV<StateSource, GCPResourceState>>> sideInputList =
Arrays.asList(KV.of((GCPResource) project, KV.of(StateSource.DESIRED, checkedPolicy)));
List<KV<GCPResource, KV<StateSource, GCPResourceState>>> mainInputList =
Arrays.asList(KV.of((GCPResource) project, KV.of(StateSource.LIVE, livePolicy)));
PCollection<KV<GCPResource, KV<StateSource, GCPResourceState>>> collection =
pipeline.apply(Create.of(sideInputList)).setCoder(
KvCoder.of(SerializableCoder.of(GCPResource.class),
KvCoder.of(SerializableCoder.of(StateSource.class),
SerializableCoder.of(GCPResourceState.class))));
PCollectionView<Map<GCPResource, KV<StateSource, GCPResourceState>>> view =
View.<GCPResource, KV<StateSource, GCPResourceState>>asMap().apply(collection);
JoinKnownGoodAndLiveStates function = new JoinKnownGoodAndLiveStates(view);
DoFnTester<KV<GCPResource, KV<StateSource, GCPResourceState>>,
KV<GCPResource, Map<StateSource, GCPResourceState>>> tester = DoFnTester.of(function);
tester.setSideInputInGlobalWindow(view, sideInputList);
List<KV<GCPResource, Map<StateSource, GCPResourceState>>> results =
tester.processBatch(mainInputList);
assertEquals(1, results.size());
}
开发者ID:GoogleCloudPlatform,项目名称:policyscanner,代码行数:28,代码来源:JoinKnownGoodAndLiveStatesTest.java
示例12: testFilterStateOneMismatch
import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Test
public void testFilterStateOneMismatch() {
GCPProject project = getSampleProject("");
GCPResourceState checkedPolicy = getSampleGCPResourcePolicy(project, 1);
GCPResourceState livePolicy = getSampleGCPResourcePolicy(project, 2);
List<KV<GCPResource, KV<StateSource, GCPResourceState>>> sideInputList =
Arrays.asList(KV.of((GCPResource) project, KV.of(StateSource.DESIRED, checkedPolicy)));
List<KV<GCPResource, KV<StateSource, GCPResourceState>>> mainInputList =
Arrays.asList(KV.of((GCPResource) project, KV.of(StateSource.LIVE, livePolicy)));
PCollection<KV<GCPResource, KV<StateSource, GCPResourceState>>> collection =
pipeline.apply(Create.of(sideInputList)).setCoder(
KvCoder.of(SerializableCoder.of(GCPResource.class),
KvCoder.of(SerializableCoder.of(StateSource.class),
SerializableCoder.of(GCPResourceState.class))));
PCollectionView<Map<GCPResource, KV<StateSource, GCPResourceState>>> view =
View.<GCPResource, KV<StateSource, GCPResourceState>>asMap().apply(collection);
JoinKnownGoodAndLiveStates function = new JoinKnownGoodAndLiveStates(view);
DoFnTester<KV<GCPResource, KV<StateSource, GCPResourceState>>,
KV<GCPResource, Map<StateSource, GCPResourceState>>> tester = DoFnTester.of(function);
tester.setSideInputInGlobalWindow(view, sideInputList);
Map<StateSource, GCPResourceState> outputMap = new HashMap<>(2);
outputMap.put(StateSource.DESIRED, checkedPolicy);
outputMap.put(StateSource.LIVE, livePolicy);
List<KV<GCPResource, Map<StateSource, GCPResourceState>>> expectedOutput =
Arrays.asList(KV.of((GCPResource) project, outputMap));
List<KV<GCPResource, Map<StateSource, GCPResourceState>>> results =
tester.processBatch(mainInputList);
assertEquals(expectedOutput, results);
}
开发者ID:GoogleCloudPlatform,项目名称:policyscanner,代码行数:34,代码来源:JoinKnownGoodAndLiveStatesTest.java
示例13: GroupIntsTest
import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Test(dataProvider = "refAndInts")
public void GroupIntsTest(Integer[] integers, Set<KV<ReferenceShard, Integer>> s) {
DoFnTester<Integer, KV<ReferenceShard, Integer>> fnTester = DoFnTester.of(new GroupIntsDoFn());
List<KV<ReferenceShard, Integer>> kvs = fnTester.processBatch(integers);
for (KV<ReferenceShard, Integer> r : kvs) {
Assert.assertTrue(s.contains(r));
}
}
开发者ID:broadinstitute,项目名称:gatk-dataflow,代码行数:9,代码来源:SampleUnitTest.java
示例14: testReadFromFileFn
import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Test
public void testReadFromFileFn(){
List<SimpleInterval> intervals = Arrays.asList(new SimpleInterval("chr7:1-202"), new SimpleInterval("chr8:2-202"));
DoFn<File, GATKRead> readfn = new DataflowUtils.LoadReadsFromFileFn(intervals, ValidationStringency.SILENT);
File inputFile = new File(getToolTestDataDir(), "example_reads.bam");
List<GATKRead> expected = getReadsFromFile(intervals, inputFile, false);
DoFnTester<File, GATKRead> tester = DoFnTester.of(readfn);
List<GATKRead> output = tester.processBatch(inputFile);
Assert.assertTrue(ReadUtils.readListsAreEqualIgnoreUUID(output, expected), "Actual reads do not match expected reads");
}
开发者ID:broadinstitute,项目名称:gatk-dataflow,代码行数:14,代码来源:DataflowUtilsUnitTest.java
示例15: setUp
import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Before
public void setUp() {
this.tester = DoFnTester.of(new FileToState());
}
开发者ID:GoogleCloudPlatform,项目名称:policyscanner,代码行数:5,代码来源:FileToStateTest.java
示例16: setUp
import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Before
public void setUp() {
tester = DoFnTester.of(new FilePathToLiveState());
GCPProject.setProjectsApiStub(projects);
}
开发者ID:GoogleCloudPlatform,项目名称:policyscanner,代码行数:6,代码来源:FilePathToLiveStateTest.java
示例17: setUp
import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Before
public void setUp() {
this.tester = DoFnTester.of(new StateDiscrepancyMessenger());
}
开发者ID:GoogleCloudPlatform,项目名称:policyscanner,代码行数:5,代码来源:StateDiscrepancyMessengerTest.java
示例18: setUp
import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Before
public void setUp() {
this.tester = DoFnTester.of(new UnmatchedStatesMessenger());
}
开发者ID:GoogleCloudPlatform,项目名称:policyscanner,代码行数:5,代码来源:UnmatchedStatesMessengerTest.java
示例19: setUp
import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Before
public void setUp() {
this.checkedTester = DoFnTester.of(new TagStateWithSource(StateSource.DESIRED));
this.liveTester = DoFnTester.of(new TagStateWithSource(StateSource.LIVE));
}
开发者ID:GoogleCloudPlatform,项目名称:policyscanner,代码行数:6,代码来源:TagStateWithSourceTest.java
示例20: setUp
import com.google.cloud.dataflow.sdk.transforms.DoFnTester; //导入依赖的package包/类
@Before
public void setUp() {
tester = DoFnTester.of(new FilePathFromPair());
}
开发者ID:GoogleCloudPlatform,项目名称:policyscanner,代码行数:5,代码来源:FilePathFromPairTest.java
注:本文中的com.google.cloud.dataflow.sdk.transforms.DoFnTester类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。 |
请发表评论