本文整理汇总了Java中org.elasticsearch.ingest.Processor类的典型用法代码示例。如果您正苦于以下问题:Java Processor类的具体用法?Java Processor怎么用?Java Processor使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。
Processor类属于org.elasticsearch.ingest包,在下文中一共展示了Processor类的20个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。
示例1: create
import org.elasticsearch.ingest.Processor; //导入依赖的package包/类
@Override
public Processor create(Map<String, Processor.Factory> processorFactories, String tag,
Map<String, Object> config) throws Exception {
String field = ConfigurationUtils.readStringProperty(TYPE, tag, config, "field");
if (field.contains(".") == false) {
throw ConfigurationUtils.newConfigurationException(ConfigurationUtils.TAG_KEY, tag, "field",
"field does not contain a dot");
}
if (field.indexOf('.') == 0 || field.lastIndexOf('.') == field.length() - 1) {
throw ConfigurationUtils.newConfigurationException(ConfigurationUtils.TAG_KEY, tag, "field",
"Field can't start or end with a dot");
}
int firstIndex = -1;
for (int index = field.indexOf('.'); index != -1; index = field.indexOf('.', index + 1)) {
if (index - firstIndex == 1) {
throw ConfigurationUtils.newConfigurationException(ConfigurationUtils.TAG_KEY, tag, "field",
"No space between dots");
}
firstIndex = index;
}
String path = ConfigurationUtils.readOptionalStringProperty(TYPE, tag, config, "path");
return new DotExpanderProcessor(tag, path, field);
}
开发者ID:justor,项目名称:elasticsearch_my,代码行数:25,代码来源:DotExpanderProcessor.java
示例2: create
import org.elasticsearch.ingest.Processor; //导入依赖的package包/类
@Override
public GrokProcessor create(Map<String, Processor.Factory> registry, String processorTag,
Map<String, Object> config) throws Exception {
String matchField = ConfigurationUtils.readStringProperty(TYPE, processorTag, config, "field");
List<String> matchPatterns = ConfigurationUtils.readList(TYPE, processorTag, config, "patterns");
boolean traceMatch = ConfigurationUtils.readBooleanProperty(TYPE, processorTag, config, "trace_match", false);
boolean ignoreMissing = ConfigurationUtils.readBooleanProperty(TYPE, processorTag, config, "ignore_missing", false);
if (matchPatterns.isEmpty()) {
throw newConfigurationException(TYPE, processorTag, "patterns", "List of patterns must not be empty");
}
Map<String, String> customPatternBank = ConfigurationUtils.readOptionalMap(TYPE, processorTag, config, "pattern_definitions");
Map<String, String> patternBank = new HashMap<>(builtinPatterns);
if (customPatternBank != null) {
patternBank.putAll(customPatternBank);
}
try {
return new GrokProcessor(processorTag, patternBank, matchPatterns, matchField, traceMatch, ignoreMissing);
} catch (Exception e) {
throw newConfigurationException(TYPE, processorTag, "patterns",
"Invalid regex pattern found in: " + matchPatterns + ". " + e.getMessage());
}
}
开发者ID:justor,项目名称:elasticsearch_my,代码行数:26,代码来源:GrokProcessor.java
示例3: create
import org.elasticsearch.ingest.Processor; //导入依赖的package包/类
@Override
public JsonProcessor create(Map<String, Processor.Factory> registry, String processorTag,
Map<String, Object> config) throws Exception {
String field = ConfigurationUtils.readStringProperty(TYPE, processorTag, config, "field");
String targetField = ConfigurationUtils.readOptionalStringProperty(TYPE, processorTag, config, "target_field");
boolean addToRoot = ConfigurationUtils.readBooleanProperty(TYPE, processorTag, config, "add_to_root", false);
if (addToRoot && targetField != null) {
throw newConfigurationException(TYPE, processorTag, "target_field",
"Cannot set a target field while also setting `add_to_root` to true");
}
if (targetField == null) {
targetField = field;
}
return new JsonProcessor(processorTag, field, targetField, addToRoot);
}
开发者ID:justor,项目名称:elasticsearch_my,代码行数:19,代码来源:JsonProcessor.java
示例4: create
import org.elasticsearch.ingest.Processor; //导入依赖的package包/类
@SuppressWarnings("unchecked")
public DateProcessor create(Map<String, Processor.Factory> registry, String processorTag,
Map<String, Object> config) throws Exception {
String field = ConfigurationUtils.readStringProperty(TYPE, processorTag, config, "field");
String targetField = ConfigurationUtils.readStringProperty(TYPE, processorTag, config, "target_field", DEFAULT_TARGET_FIELD);
String timezoneString = ConfigurationUtils.readOptionalStringProperty(TYPE, processorTag, config, "timezone");
DateTimeZone timezone = timezoneString == null ? DateTimeZone.UTC : DateTimeZone.forID(timezoneString);
String localeString = ConfigurationUtils.readOptionalStringProperty(TYPE, processorTag, config, "locale");
Locale locale = Locale.ENGLISH;
if (localeString != null) {
try {
locale = (new Locale.Builder()).setLanguageTag(localeString).build();
} catch (IllformedLocaleException e) {
throw new IllegalArgumentException("Invalid language tag specified: " + localeString);
}
}
List<String> formats = ConfigurationUtils.readList(TYPE, processorTag, config, "formats");
return new DateProcessor(processorTag, timezone, locale, field, formats, targetField);
}
开发者ID:justor,项目名称:elasticsearch_my,代码行数:20,代码来源:DateProcessor.java
示例5: create
import org.elasticsearch.ingest.Processor; //导入依赖的package包/类
@Override
public SortProcessor create(Map<String, Processor.Factory> registry, String processorTag,
Map<String, Object> config) throws Exception {
String field = ConfigurationUtils.readStringProperty(TYPE, processorTag, config, FIELD);
try {
SortOrder direction = SortOrder.fromString(
ConfigurationUtils.readStringProperty(
TYPE,
processorTag,
config,
ORDER,
DEFAULT_ORDER));
return new SortProcessor(processorTag, field, direction);
} catch (IllegalArgumentException e) {
throw ConfigurationUtils.newConfigurationException(TYPE, processorTag, ORDER, e.getMessage());
}
}
开发者ID:justor,项目名称:elasticsearch_my,代码行数:18,代码来源:SortProcessor.java
示例6: testJoinStrings
import org.elasticsearch.ingest.Processor; //导入依赖的package包/类
public void testJoinStrings() throws Exception {
IngestDocument ingestDocument = RandomDocumentPicks.randomIngestDocument(random());
int numItems = randomIntBetween(1, 10);
String separator = randomFrom(SEPARATORS);
List<String> fieldValue = new ArrayList<>(numItems);
String expectedResult = "";
for (int j = 0; j < numItems; j++) {
String value = randomAsciiOfLengthBetween(1, 10);
fieldValue.add(value);
expectedResult += value;
if (j < numItems - 1) {
expectedResult += separator;
}
}
String fieldName = RandomDocumentPicks.addRandomField(random(), ingestDocument, fieldValue);
Processor processor = new JoinProcessor(randomAsciiOfLength(10), fieldName, separator);
processor.execute(ingestDocument);
assertThat(ingestDocument.getFieldValue(fieldName, String.class), equalTo(expectedResult));
}
开发者ID:justor,项目名称:elasticsearch_my,代码行数:20,代码来源:JoinProcessorTests.java
示例7: testJoinIntegers
import org.elasticsearch.ingest.Processor; //导入依赖的package包/类
public void testJoinIntegers() throws Exception {
IngestDocument ingestDocument = RandomDocumentPicks.randomIngestDocument(random());
int numItems = randomIntBetween(1, 10);
String separator = randomFrom(SEPARATORS);
List<Integer> fieldValue = new ArrayList<>(numItems);
String expectedResult = "";
for (int j = 0; j < numItems; j++) {
int value = randomInt();
fieldValue.add(value);
expectedResult += value;
if (j < numItems - 1) {
expectedResult += separator;
}
}
String fieldName = RandomDocumentPicks.addRandomField(random(), ingestDocument, fieldValue);
Processor processor = new JoinProcessor(randomAsciiOfLength(10), fieldName, separator);
processor.execute(ingestDocument);
assertThat(ingestDocument.getFieldValue(fieldName, String.class), equalTo(expectedResult));
}
开发者ID:justor,项目名称:elasticsearch_my,代码行数:20,代码来源:JoinProcessorTests.java
示例8: testSplitAppendable
import org.elasticsearch.ingest.Processor; //导入依赖的package包/类
public void testSplitAppendable() throws Exception {
Map<String, Object> splitConfig = new HashMap<>();
splitConfig.put("field", "flags");
splitConfig.put("separator", "\\|");
Processor splitProcessor = (new SplitProcessor.Factory()).create(null, null, splitConfig);
Map<String, Object> source = new HashMap<>();
source.put("flags", "new|hot|super|fun|interesting");
IngestDocument ingestDocument = new IngestDocument(source, new HashMap<>());
splitProcessor.execute(ingestDocument);
@SuppressWarnings("unchecked")
List<String> flags = (List<String>)ingestDocument.getFieldValue("flags", List.class);
assertThat(flags, equalTo(Arrays.asList("new", "hot", "super", "fun", "interesting")));
ingestDocument.appendFieldValue("flags", "additional_flag");
assertThat(ingestDocument.getFieldValue("flags", List.class), equalTo(Arrays.asList("new", "hot", "super",
"fun", "interesting", "additional_flag")));
}
开发者ID:justor,项目名称:elasticsearch_my,代码行数:17,代码来源:SplitProcessorTests.java
示例9: testCreateWithTooManyProcessorTypes
import org.elasticsearch.ingest.Processor; //导入依赖的package包/类
public void testCreateWithTooManyProcessorTypes() throws Exception {
Processor processor = new TestProcessor(ingestDocument -> { });
Map<String, Processor.Factory> registry = new HashMap<>();
registry.put("_first", (r, t, c) -> processor);
registry.put("_second", (r, t, c) -> processor);
ForEachProcessor.Factory forEachFactory = new ForEachProcessor.Factory();
Map<String, Object> config = new HashMap<>();
config.put("field", "_field");
Map<String, Object> processorTypes = new HashMap<>();
processorTypes.put("_first", Collections.emptyMap());
processorTypes.put("_second", Collections.emptyMap());
config.put("processor", processorTypes);
Exception exception = expectThrows(ElasticsearchParseException.class, () -> forEachFactory.create(registry, null, config));
assertThat(exception.getMessage(), equalTo("[processor] Must specify exactly one processor type"));
}
开发者ID:justor,项目名称:elasticsearch_my,代码行数:17,代码来源:ForEachProcessorFactoryTests.java
示例10: testConvertBooleanList
import org.elasticsearch.ingest.Processor; //导入依赖的package包/类
public void testConvertBooleanList() throws Exception {
IngestDocument ingestDocument = RandomDocumentPicks.randomIngestDocument(random());
int numItems = randomIntBetween(1, 10);
List<String> fieldValue = new ArrayList<>();
List<Boolean> expectedList = new ArrayList<>();
for (int j = 0; j < numItems; j++) {
boolean randomBoolean = randomBoolean();
String booleanString = Boolean.toString(randomBoolean);
if (randomBoolean) {
booleanString = booleanString.toUpperCase(Locale.ROOT);
}
fieldValue.add(booleanString);
expectedList.add(randomBoolean);
}
String fieldName = RandomDocumentPicks.addRandomField(random(), ingestDocument, fieldValue);
Processor processor = new ConvertProcessor(randomAsciiOfLength(10), fieldName, fieldName, Type.BOOLEAN, false);
processor.execute(ingestDocument);
assertThat(ingestDocument.getFieldValue(fieldName, List.class), equalTo(expectedList));
}
开发者ID:justor,项目名称:elasticsearch_my,代码行数:20,代码来源:ConvertProcessorTests.java
示例11: getProcessors
import org.elasticsearch.ingest.Processor; //导入依赖的package包/类
@Override
public Map<String, Processor.Factory> getProcessors(Processor.Parameters parameters) {
Path userAgentConfigDirectory = parameters.env.configFile().resolve("ingest-user-agent");
if (Files.exists(userAgentConfigDirectory) == false && Files.isDirectory(userAgentConfigDirectory)) {
throw new IllegalStateException(
"the user agent directory [" + userAgentConfigDirectory + "] containing the regex file doesn't exist");
}
long cacheSize = CACHE_SIZE_SETTING.get(parameters.env.settings());
Map<String, UserAgentParser> userAgentParsers;
try {
userAgentParsers = createUserAgentParsers(userAgentConfigDirectory, new UserAgentCache(cacheSize));
} catch (IOException e) {
throw new RuntimeException(e);
}
return Collections.singletonMap(UserAgentProcessor.TYPE, new UserAgentProcessor.Factory(userAgentParsers));
}
开发者ID:justor,项目名称:elasticsearch_my,代码行数:19,代码来源:IngestUserAgentPlugin.java
示例12: getProcessors
import org.elasticsearch.ingest.Processor; //导入依赖的package包/类
@Override
public Map<String, Processor.Factory> getProcessors(Processor.Parameters parameters) {
if (databaseReaders != null) {
throw new IllegalStateException("getProcessors called twice for geoip plugin!!");
}
Path geoIpConfigDirectory = parameters.env.configFile().resolve("ingest-geoip");
NodeCache cache;
long cacheSize = CACHE_SIZE.get(parameters.env.settings());
if (cacheSize > 0) {
cache = new GeoIpCache(cacheSize);
} else {
cache = NoCache.getInstance();
}
try {
databaseReaders = loadDatabaseReaders(geoIpConfigDirectory, cache);
} catch (IOException e) {
throw new RuntimeException(e);
}
return Collections.singletonMap(GeoIpProcessor.TYPE, new GeoIpProcessor.Factory(databaseReaders));
}
开发者ID:justor,项目名称:elasticsearch_my,代码行数:21,代码来源:IngestGeoIpPlugin.java
示例13: testRenameAtomicOperationRemoveFails
import org.elasticsearch.ingest.Processor; //导入依赖的package包/类
public void testRenameAtomicOperationRemoveFails() throws Exception {
Map<String, Object> source = new HashMap<String, Object>() {
@Override
public Object remove(Object key) {
if (key.equals("list")) {
throw new UnsupportedOperationException();
}
return super.remove(key);
}
};
source.put("list", Collections.singletonList("item"));
IngestDocument ingestDocument = new IngestDocument(source, Collections.emptyMap());
Processor processor = new RenameProcessor(randomAsciiOfLength(10), "list", "new_field", false);
try {
processor.execute(ingestDocument);
fail("processor execute should have failed");
} catch (UnsupportedOperationException e) {
//the set failed, the old field has not been removed
assertThat(ingestDocument.getSourceAndMetadata().containsKey("list"), equalTo(true));
assertThat(ingestDocument.getSourceAndMetadata().containsKey("new_field"), equalTo(false));
}
}
开发者ID:justor,项目名称:elasticsearch_my,代码行数:24,代码来源:RenameProcessorTests.java
示例14: create
import org.elasticsearch.ingest.Processor; //导入依赖的package包/类
@Override
public CsvProcessor create(Map<String, Processor.Factory> factories, String tag, Map<String, Object> config)
throws Exception {
String field = readStringProperty(TYPE, tag, config, "field");
List<String> columns = readList(TYPE, tag, config, "columns");
// FIXME should test duplicate name
if (columns.size() == 0) {
throw new IllegalArgumentException("columns is missing");
}
String quoteChar = readStringProperty(TYPE, tag, config, "quote_char", "\"");
if (Strings.isEmpty(quoteChar) || quoteChar.length() != 1) {
throw new IllegalArgumentException("quote_char must be a character, like \" or \'");
}
String separator = readStringProperty(TYPE, tag, config, "separator", ",");
if (Strings.isEmpty(separator) || separator.length() != 1) {
throw new IllegalArgumentException("separator must be a character, like , or TAB");
}
return new CsvProcessor(tag, field, columns, quoteChar.charAt(0), separator.charAt(0));
}
开发者ID:johtani,项目名称:elasticsearch-ingest-csv,代码行数:21,代码来源:CsvProcessor.java
示例15: testSortStrings
import org.elasticsearch.ingest.Processor; //导入依赖的package包/类
public void testSortStrings() throws Exception {
IngestDocument ingestDocument = RandomDocumentPicks.randomIngestDocument(random());
int numItems = randomIntBetween(1, 10);
List<String> fieldValue = new ArrayList<>(numItems);
List<String> expectedResult = new ArrayList<>(numItems);
for (int j = 0; j < numItems; j++) {
String value = randomAsciiOfLengthBetween(1, 10);
fieldValue.add(value);
expectedResult.add(value);
}
Collections.sort(expectedResult);
SortOrder order = randomBoolean() ? SortOrder.ASCENDING : SortOrder.DESCENDING;
if (order.equals(SortOrder.DESCENDING)) {
Collections.reverse(expectedResult);
}
String fieldName = RandomDocumentPicks.addRandomField(random(), ingestDocument, fieldValue);
Processor processor = new SortProcessor(randomAsciiOfLength(10), fieldName, order);
processor.execute(ingestDocument);
assertEquals(ingestDocument.getFieldValue(fieldName, List.class), expectedResult);
}
开发者ID:justor,项目名称:elasticsearch_my,代码行数:23,代码来源:SortProcessorTests.java
示例16: testSortIntegers
import org.elasticsearch.ingest.Processor; //导入依赖的package包/类
public void testSortIntegers() throws Exception {
IngestDocument ingestDocument = RandomDocumentPicks.randomIngestDocument(random());
int numItems = randomIntBetween(1, 10);
List<Integer> fieldValue = new ArrayList<>(numItems);
List<Integer> expectedResult = new ArrayList<>(numItems);
for (int j = 0; j < numItems; j++) {
Integer value = randomIntBetween(1, 100);
fieldValue.add(value);
expectedResult.add(value);
}
Collections.sort(expectedResult);
SortOrder order = randomBoolean() ? SortOrder.ASCENDING : SortOrder.DESCENDING;
if (order.equals(SortOrder.DESCENDING)) {
Collections.reverse(expectedResult);
}
String fieldName = RandomDocumentPicks.addRandomField(random(), ingestDocument, fieldValue);
Processor processor = new SortProcessor(randomAsciiOfLength(10), fieldName, order);
processor.execute(ingestDocument);
assertEquals(ingestDocument.getFieldValue(fieldName, List.class), expectedResult);
}
开发者ID:justor,项目名称:elasticsearch_my,代码行数:23,代码来源:SortProcessorTests.java
示例17: testSortShorts
import org.elasticsearch.ingest.Processor; //导入依赖的package包/类
public void testSortShorts() throws Exception {
IngestDocument ingestDocument = RandomDocumentPicks.randomIngestDocument(random());
int numItems = randomIntBetween(1, 10);
List<Short> fieldValue = new ArrayList<>(numItems);
List<Short> expectedResult = new ArrayList<>(numItems);
for (int j = 0; j < numItems; j++) {
Short value = randomShort();
fieldValue.add(value);
expectedResult.add(value);
}
Collections.sort(expectedResult);
SortOrder order = randomBoolean() ? SortOrder.ASCENDING : SortOrder.DESCENDING;
if (order.equals(SortOrder.DESCENDING)) {
Collections.reverse(expectedResult);
}
String fieldName = RandomDocumentPicks.addRandomField(random(), ingestDocument, fieldValue);
Processor processor = new SortProcessor(randomAsciiOfLength(10), fieldName, order);
processor.execute(ingestDocument);
assertEquals(ingestDocument.getFieldValue(fieldName, List.class), expectedResult);
}
开发者ID:justor,项目名称:elasticsearch_my,代码行数:23,代码来源:SortProcessorTests.java
示例18: testSortFloats
import org.elasticsearch.ingest.Processor; //导入依赖的package包/类
public void testSortFloats() throws Exception {
IngestDocument ingestDocument = RandomDocumentPicks.randomIngestDocument(random());
int numItems = randomIntBetween(1, 10);
List<Float> fieldValue = new ArrayList<>(numItems);
List<Float> expectedResult = new ArrayList<>(numItems);
for (int j = 0; j < numItems; j++) {
Float value = randomFloat();
fieldValue.add(value);
expectedResult.add(value);
}
Collections.sort(expectedResult);
SortOrder order = randomBoolean() ? SortOrder.ASCENDING : SortOrder.DESCENDING;
if (order.equals(SortOrder.DESCENDING)) {
Collections.reverse(expectedResult);
}
String fieldName = RandomDocumentPicks.addRandomField(random(), ingestDocument, fieldValue);
Processor processor = new SortProcessor(randomAsciiOfLength(10), fieldName, order);
processor.execute(ingestDocument);
assertEquals(ingestDocument.getFieldValue(fieldName, List.class), expectedResult);
}
开发者ID:justor,项目名称:elasticsearch_my,代码行数:23,代码来源:SortProcessorTests.java
示例19: testSortBytes
import org.elasticsearch.ingest.Processor; //导入依赖的package包/类
public void testSortBytes() throws Exception {
IngestDocument ingestDocument = RandomDocumentPicks.randomIngestDocument(random());
int numItems = randomIntBetween(1, 10);
List<Byte> fieldValue = new ArrayList<>(numItems);
List<Byte> expectedResult = new ArrayList<>(numItems);
for (int j = 0; j < numItems; j++) {
Byte value = randomByte();
fieldValue.add(value);
expectedResult.add(value);
}
Collections.sort(expectedResult);
SortOrder order = randomBoolean() ? SortOrder.ASCENDING : SortOrder.DESCENDING;
if (order.equals(SortOrder.DESCENDING)) {
Collections.reverse(expectedResult);
}
String fieldName = RandomDocumentPicks.addRandomField(random(), ingestDocument, fieldValue);
Processor processor = new SortProcessor(randomAsciiOfLength(10), fieldName, order);
processor.execute(ingestDocument);
assertEquals(ingestDocument.getFieldValue(fieldName, List.class), expectedResult);
}
开发者ID:justor,项目名称:elasticsearch_my,代码行数:23,代码来源:SortProcessorTests.java
示例20: create
import org.elasticsearch.ingest.Processor; //导入依赖的package包/类
@Override
public AttachmentProcessor create(Map<String, Processor.Factory> registry, String processorTag,
Map<String, Object> config) throws Exception {
String field = readStringProperty(TYPE, processorTag, config, "field");
String targetField = readStringProperty(TYPE, processorTag, config, "target_field", "attachment");
List<String> properyNames = readOptionalList(TYPE, processorTag, config, "properties");
int indexedChars = readIntProperty(TYPE, processorTag, config, "indexed_chars", NUMBER_OF_CHARS_INDEXED);
boolean ignoreMissing = readBooleanProperty(TYPE, processorTag, config, "ignore_missing", false);
final Set<Property> properties;
if (properyNames != null) {
properties = EnumSet.noneOf(Property.class);
for (String fieldName : properyNames) {
try {
properties.add(Property.parse(fieldName));
} catch (Exception e) {
throw newConfigurationException(TYPE, processorTag, "properties", "illegal field option [" +
fieldName + "]. valid values are " + Arrays.toString(Property.values()));
}
}
} else {
properties = DEFAULT_PROPERTIES;
}
return new AttachmentProcessor(processorTag, field, targetField, properties, indexedChars, ignoreMissing);
}
开发者ID:justor,项目名称:elasticsearch_my,代码行数:27,代码来源:AttachmentProcessor.java
注:本文中的org.elasticsearch.ingest.Processor类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。 |
请发表评论