本文整理汇总了Java中backtype.storm.transactional.TransactionAttempt类的典型用法代码示例。如果您正苦于以下问题:Java TransactionAttempt类的具体用法?Java TransactionAttempt怎么用?Java TransactionAttempt使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。
TransactionAttempt类属于backtype.storm.transactional包,在下文中一共展示了TransactionAttempt类的20个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。
示例1: execute
import backtype.storm.transactional.TransactionAttempt; //导入依赖的package包/类
@Override
public void execute(Tuple tuple, BasicOutputCollector collector) {
TransactionAttempt attempt = (TransactionAttempt) tuple.getValue(0);
int curr = tuple.getInteger(2);
Integer prev = tuple.getInteger(3);
int currBucket = curr / BUCKET_SIZE;
Integer prevBucket = null;
if (prev != null) {
prevBucket = prev / BUCKET_SIZE;
}
if (prevBucket == null) {
collector.emit(new Values(attempt, currBucket, 1));
} else if (currBucket != prevBucket) {
collector.emit(new Values(attempt, currBucket, 1));
collector.emit(new Values(attempt, prevBucket, -1));
}
}
开发者ID:zhangjunfang,项目名称:jstorm-0.9.6.3-,代码行数:20,代码来源:TransactionalWordsTest.java
示例2: execute
import backtype.storm.transactional.TransactionAttempt; //导入依赖的package包/类
@Override
public void execute(Tuple tuple, BasicOutputCollector collector) {
TransactionAttempt attempt = (TransactionAttempt) tuple.getValue(0);
int curr = tuple.getInteger(2);
Integer prev = tuple.getInteger(3);
int currBucket = curr / BUCKET_SIZE;
Integer prevBucket = null;
if (prev != null) {
prevBucket = prev / BUCKET_SIZE;
}
if (prevBucket == null) {
collector.emit(new Values(attempt, currBucket, 1));
}
else if (currBucket != prevBucket) {
collector.emit(new Values(attempt, currBucket, 1));
collector.emit(new Values(attempt, prevBucket, -1));
}
}
开发者ID:luozhaoyu,项目名称:big-data-system,代码行数:21,代码来源:TransactionalWords.java
示例3: emitBatch
import backtype.storm.transactional.TransactionAttempt; //导入依赖的package包/类
@Override
public void emitBatch(final TransactionAttempt tx, final Integer partitions, final BatchOutputCollector collector) {
for (int i = _index; i < partitions; i += _numTasks) {
if (!_partitionStates.containsKey(i)) {
_partitionStates.put(i, new RotatingTransactionalState(_state, "" + i));
}
RotatingTransactionalState state = _partitionStates.get(i);
final int partition = i;
Object meta = state.getStateOrCreate(tx.getTransactionId(), new RotatingTransactionalState.StateInitializer() {
@Override
public Object init(BigInteger txid, Object lastState) {
return _emitter.emitPartitionBatchNew(tx, collector, partition, lastState);
}
});
// it's null if one of:
// a) a later transaction batch was emitted before this, so we should skip this batch
// b) if didn't exist and was created (in which case the StateInitializer was invoked and
// it was emitted
if (meta != null) {
_emitter.emitPartitionBatch(tx, collector, partition, meta);
}
}
}
开发者ID:kkllwww007,项目名称:jstrom,代码行数:25,代码来源:PartitionedTransactionalSpoutExecutor.java
示例4: execute
import backtype.storm.transactional.TransactionAttempt; //导入依赖的package包/类
@Override
public void execute(Tuple tuple, BasicOutputCollector collector) {
TransactionAttempt attempt = (TransactionAttempt) tuple.getValue(0);
int curr = tuple.getInteger(2);
Integer prev = tuple.getInteger(3);
int currBucket = curr / BUCKET_SIZE;
Integer prevBucket = null;
if (prev != null) {
prevBucket = prev / BUCKET_SIZE;
}
if (prevBucket == null) {
collector.emit(new Values(attempt, currBucket, 1));
} else if (currBucket != prevBucket) {
collector.emit(new Values(attempt, currBucket, 1));
collector.emit(new Values(attempt, prevBucket, -1));
}
}
开发者ID:kkllwww007,项目名称:jstrom,代码行数:20,代码来源:TransactionalWordsTest.java
示例5: emitBatch
import backtype.storm.transactional.TransactionAttempt; //导入依赖的package包/类
@Override
public void emitBatch(final TransactionAttempt tx, final Integer partitions,
final BatchOutputCollector collector) {
for(int i=_index; i < partitions; i+=_numTasks) {
if(!_partitionStates.containsKey(i)) {
_partitionStates.put(i, new RotatingTransactionalState(_state, "" + i));
}
RotatingTransactionalState state = _partitionStates.get(i);
final int partition = i;
Object meta = state.getStateOrCreate(tx.getTransactionId(),
new RotatingTransactionalState.StateInitializer() {
@Override
public Object init(BigInteger txid, Object lastState) {
return _emitter.emitPartitionBatchNew(tx, collector, partition, lastState);
}
});
// it's null if one of:
// a) a later transaction batch was emitted before this, so we should skip this batch
// b) if didn't exist and was created (in which case the StateInitializer was invoked and
// it was emitted
if(meta!=null) {
_emitter.emitPartitionBatch(tx, collector, partition, meta);
}
}
}
开发者ID:metamx,项目名称:incubator-storm,代码行数:27,代码来源:PartitionedTransactionalSpoutExecutor.java
示例6: execute
import backtype.storm.transactional.TransactionAttempt; //导入依赖的package包/类
@Override
public void execute(Tuple tuple, BasicOutputCollector collector) {
TransactionAttempt attempt = (TransactionAttempt) tuple.getValue(0);
int curr = tuple.getInteger(2);
Integer prev = tuple.getInteger(3);
int currBucket = curr / BUCKET_SIZE;
Integer prevBucket = null;
if(prev!=null) {
prevBucket = prev / BUCKET_SIZE;
}
if(prevBucket==null) {
collector.emit(new Values(attempt, currBucket, 1));
} else if(currBucket != prevBucket) {
collector.emit(new Values(attempt, currBucket, 1));
collector.emit(new Values(attempt, prevBucket, -1));
}
}
开发者ID:adarshms,项目名称:sentweet,代码行数:20,代码来源:TransactionalWords.java
示例7: emitPartitionBatch
import backtype.storm.transactional.TransactionAttempt; //导入依赖的package包/类
@Override
public BatchMeta emitPartitionBatch(TransactionAttempt attempt, BatchOutputCollector collector, int partition, BatchMeta lastMeta) {
try {
SimpleConsumer consumer = _connections.getConsumer(partition);
return KafkaUtils.emitPartitionBatchNew(_config, partition, consumer, attempt, collector, lastMeta);
} catch(FailedFetchException e) {
LOG.warn("Failed to fetch from partition " + partition);
if(lastMeta==null) {
return null;
} else {
BatchMeta ret = new BatchMeta();
ret.offset = lastMeta.nextOffset;
ret.nextOffset = lastMeta.nextOffset;
return ret;
}
}
}
开发者ID:YinYanfei,项目名称:CadalWorkspace,代码行数:18,代码来源:OpaqueTransactionalKafkaSpout.java
示例8: emitPartitionBatchNew
import backtype.storm.transactional.TransactionAttempt; //导入依赖的package包/类
public static BatchMeta emitPartitionBatchNew(KafkaConfig config, int partition, SimpleConsumer consumer, TransactionAttempt attempt, BatchOutputCollector collector, BatchMeta lastMeta) {
long offset = 0;
if(lastMeta!=null) {
offset = lastMeta.nextOffset;
}
ByteBufferMessageSet msgs;
try {
msgs = consumer.fetch(new FetchRequest(config.topic, partition % config.partitionsPerHost, offset, config.fetchSizeBytes));
} catch(Exception e) {
if(e instanceof ConnectException) {
throw new FailedFetchException(e);
} else {
throw new RuntimeException(e);
}
}
long endoffset = offset;
for(MessageAndOffset msg: msgs) {
emit(config, attempt, collector, msg.message());
endoffset = msg.offset();
}
BatchMeta newMeta = new BatchMeta();
newMeta.offset = offset;
newMeta.nextOffset = endoffset;
return newMeta;
}
开发者ID:YinYanfei,项目名称:CadalWorkspace,代码行数:26,代码来源:KafkaUtils.java
示例9: execute
import backtype.storm.transactional.TransactionAttempt; //导入依赖的package包/类
@Override
public void execute(Tuple tuple, BasicOutputCollector collector) {
TransactionAttempt attempt = (TransactionAttempt) tuple.getValue(0);
int curr = tuple.getInteger(2);
Integer prev = tuple.getInteger(3);
int currBucket = curr / BUCKET_SIZE;
Integer prevBucket = null;
if (prev != null) {
prevBucket = prev / BUCKET_SIZE;
}
if (prevBucket == null) {
collector.emit(new Values(attempt, currBucket, 1));
} else if (currBucket != prevBucket) {
collector.emit(new Values(attempt, currBucket, 1));
collector.emit(new Values(attempt, prevBucket, -1));
}
}
开发者ID:alibaba,项目名称:jstorm,代码行数:20,代码来源:TransactionalWords.java
示例10: commit
import backtype.storm.transactional.TransactionAttempt; //导入依赖的package包/类
@Override
public void commit(TransactionAttempt attempt) {
BigInteger txid = attempt.getTransactionId();
Map<Integer, Object> metas = _cachedMetas.remove(txid);
for (Integer partition : metas.keySet()) {
Object meta = metas.get(partition);
_partitionStates.get(partition).overrideState(txid, meta);
}
}
开发者ID:zhangjunfang,项目名称:jstorm-0.9.6.3-,代码行数:10,代码来源:OpaquePartitionedTransactionalSpoutExecutor.java
示例11: emitBatch
import backtype.storm.transactional.TransactionAttempt; //导入依赖的package包/类
@Override
public void emitBatch(final TransactionAttempt tx,
final Integer partitions, final BatchOutputCollector collector) {
for (int i = _index; i < partitions; i += _numTasks) {
if (!_partitionStates.containsKey(i)) {
_partitionStates.put(i, new RotatingTransactionalState(
_state, "" + i));
}
RotatingTransactionalState state = _partitionStates.get(i);
final int partition = i;
Object meta = state.getStateOrCreate(tx.getTransactionId(),
new RotatingTransactionalState.StateInitializer() {
@Override
public Object init(BigInteger txid, Object lastState) {
return _emitter.emitPartitionBatchNew(tx,
collector, partition, lastState);
}
});
// it's null if one of:
// a) a later transaction batch was emitted before this, so we
// should skip this batch
// b) if didn't exist and was created (in which case the
// StateInitializer was invoked and
// it was emitted
if (meta != null) {
_emitter.emitPartitionBatch(tx, collector, partition, meta);
}
}
}
开发者ID:zhangjunfang,项目名称:jstorm-0.9.6.3-,代码行数:31,代码来源:PartitionedTransactionalSpoutExecutor.java
示例12: emitPartitionBatch
import backtype.storm.transactional.TransactionAttempt; //导入依赖的package包/类
@Override
public MemoryTransactionalSpoutMeta emitPartitionBatch(
TransactionAttempt tx, BatchOutputCollector collector,
int partition, MemoryTransactionalSpoutMeta lastPartitionMeta) {
if (!Boolean.FALSE.equals(getDisabledStatuses().get(partition))) {
int index;
if (lastPartitionMeta == null) {
index = 0;
} else {
index = lastPartitionMeta.index + lastPartitionMeta.amt;
}
List<List<Object>> queue = getQueues().get(partition);
int total = queue.size();
int left = total - index;
int toTake = Math.min(left, _takeAmt);
MemoryTransactionalSpoutMeta ret = new MemoryTransactionalSpoutMeta(
index, toTake);
for (int i = ret.index; i < ret.index + ret.amt; i++) {
List<Object> toEmit = new ArrayList<Object>(queue.get(i));
toEmit.add(0, tx);
collector.emit(toEmit);
}
if (toTake == 0) {
// this is a pretty hacky way to determine when all the
// partitions have been committed
// wait until we've emitted max-spout-pending empty
// partitions for the partition
int curr = Utils.get(_emptyPartitions, partition, 0) + 1;
_emptyPartitions.put(partition, curr);
if (curr > _maxSpoutPending) {
getFinishedStatuses().put(partition, true);
}
}
return ret;
} else {
return null;
}
}
开发者ID:zhangjunfang,项目名称:jstorm-0.9.6.3-,代码行数:40,代码来源:OpaqueMemoryTransactionalSpout.java
示例13: emitPartitionBatchNew
import backtype.storm.transactional.TransactionAttempt; //导入依赖的package包/类
@Override
public MemoryTransactionalSpoutMeta emitPartitionBatchNew(
TransactionAttempt tx, BatchOutputCollector collector,
int partition, MemoryTransactionalSpoutMeta lastPartitionMeta) {
int index;
if (lastPartitionMeta == null) {
index = 0;
} else {
index = lastPartitionMeta.index + lastPartitionMeta.amt;
}
List<List<Object>> queue = getQueues().get(partition);
int total = queue.size();
int left = total - index;
int toTake = Math.min(left, _takeAmt);
MemoryTransactionalSpoutMeta ret = new MemoryTransactionalSpoutMeta(
index, toTake);
emitPartitionBatch(tx, collector, partition, ret);
if (toTake == 0) {
// this is a pretty hacky way to determine when all the
// partitions have been committed
// wait until we've emitted max-spout-pending empty partitions
// for the partition
int curr = Utils.get(_emptyPartitions, partition, 0) + 1;
_emptyPartitions.put(partition, curr);
if (curr > _maxSpoutPending) {
Map<Integer, Boolean> finishedStatuses = getFinishedStatuses();
// will be null in remote mode
if (finishedStatuses != null) {
finishedStatuses.put(partition, true);
}
}
}
return ret;
}
开发者ID:zhangjunfang,项目名称:jstorm-0.9.6.3-,代码行数:36,代码来源:MemoryTransactionalSpout.java
示例14: emitPartitionBatch
import backtype.storm.transactional.TransactionAttempt; //导入依赖的package包/类
@Override
public void emitPartitionBatch(TransactionAttempt tx,
BatchOutputCollector collector, int partition,
MemoryTransactionalSpoutMeta partitionMeta) {
List<List<Object>> queue = getQueues().get(partition);
for (int i = partitionMeta.index; i < partitionMeta.index
+ partitionMeta.amt; i++) {
List<Object> toEmit = new ArrayList<Object>(queue.get(i));
toEmit.add(0, tx);
collector.emit(toEmit);
}
}
开发者ID:zhangjunfang,项目名称:jstorm-0.9.6.3-,代码行数:13,代码来源:MemoryTransactionalSpout.java
示例15: commit
import backtype.storm.transactional.TransactionAttempt; //导入依赖的package包/类
@Override
public void commit(TransactionAttempt attempt) {
BigInteger txid = attempt.getTransactionId();
Map<Integer, Object> metas = _cachedMetas.remove(txid);
for (Integer partition : metas.keySet()) {
Object meta = metas.get(partition);
_partitionStates.get(partition).overrideState(txid, meta);
}
}
开发者ID:kkllwww007,项目名称:jstrom,代码行数:10,代码来源:OpaquePartitionedTransactionalSpoutExecutor.java
示例16: emitPartitionBatch
import backtype.storm.transactional.TransactionAttempt; //导入依赖的package包/类
@Override
public MemoryTransactionalSpoutMeta emitPartitionBatch(TransactionAttempt tx, BatchOutputCollector collector, int partition,
MemoryTransactionalSpoutMeta lastPartitionMeta) {
if (!Boolean.FALSE.equals(getDisabledStatuses().get(partition))) {
int index;
if (lastPartitionMeta == null) {
index = 0;
} else {
index = lastPartitionMeta.index + lastPartitionMeta.amt;
}
List<List<Object>> queue = getQueues().get(partition);
int total = queue.size();
int left = total - index;
int toTake = Math.min(left, _takeAmt);
MemoryTransactionalSpoutMeta ret = new MemoryTransactionalSpoutMeta(index, toTake);
for (int i = ret.index; i < ret.index + ret.amt; i++) {
List<Object> toEmit = new ArrayList<Object>(queue.get(i));
toEmit.add(0, tx);
collector.emit(toEmit);
}
if (toTake == 0) {
// this is a pretty hacky way to determine when all the partitions have been committed
// wait until we've emitted max-spout-pending empty partitions for the partition
int curr = Utils.get(_emptyPartitions, partition, 0) + 1;
_emptyPartitions.put(partition, curr);
if (curr > _maxSpoutPending) {
getFinishedStatuses().put(partition, true);
}
}
return ret;
} else {
return null;
}
}
开发者ID:kkllwww007,项目名称:jstrom,代码行数:36,代码来源:OpaqueMemoryTransactionalSpout.java
示例17: emitPartitionBatchNew
import backtype.storm.transactional.TransactionAttempt; //导入依赖的package包/类
@Override
public MemoryTransactionalSpoutMeta emitPartitionBatchNew(TransactionAttempt tx, BatchOutputCollector collector, int partition,
MemoryTransactionalSpoutMeta lastPartitionMeta) {
int index;
if (lastPartitionMeta == null) {
index = 0;
} else {
index = lastPartitionMeta.index + lastPartitionMeta.amt;
}
List<List<Object>> queue = getQueues().get(partition);
int total = queue.size();
int left = total - index;
int toTake = Math.min(left, _takeAmt);
MemoryTransactionalSpoutMeta ret = new MemoryTransactionalSpoutMeta(index, toTake);
emitPartitionBatch(tx, collector, partition, ret);
if (toTake == 0) {
// this is a pretty hacky way to determine when all the partitions have been committed
// wait until we've emitted max-spout-pending empty partitions for the partition
int curr = Utils.get(_emptyPartitions, partition, 0) + 1;
_emptyPartitions.put(partition, curr);
if (curr > _maxSpoutPending) {
Map<Integer, Boolean> finishedStatuses = getFinishedStatuses();
// will be null in remote mode
if (finishedStatuses != null) {
finishedStatuses.put(partition, true);
}
}
}
return ret;
}
开发者ID:kkllwww007,项目名称:jstrom,代码行数:32,代码来源:MemoryTransactionalSpout.java
示例18: emitPartitionBatch
import backtype.storm.transactional.TransactionAttempt; //导入依赖的package包/类
@Override
public void emitPartitionBatch(TransactionAttempt tx, BatchOutputCollector collector, int partition, MemoryTransactionalSpoutMeta partitionMeta) {
List<List<Object>> queue = getQueues().get(partition);
for (int i = partitionMeta.index; i < partitionMeta.index + partitionMeta.amt; i++) {
List<Object> toEmit = new ArrayList<Object>(queue.get(i));
toEmit.add(0, tx);
collector.emit(toEmit);
}
}
开发者ID:kkllwww007,项目名称:jstrom,代码行数:10,代码来源:MemoryTransactionalSpout.java
示例19: commit
import backtype.storm.transactional.TransactionAttempt; //导入依赖的package包/类
@Override
public void commit(TransactionAttempt attempt) {
BigInteger txid = attempt.getTransactionId();
Map<Integer, Object> metas = _cachedMetas.remove(txid);
for(Integer partition: metas.keySet()) {
Object meta = metas.get(partition);
_partitionStates.get(partition).overrideState(txid, meta);
}
}
开发者ID:metamx,项目名称:incubator-storm,代码行数:10,代码来源:OpaquePartitionedTransactionalSpoutExecutor.java
示例20: emitPartitionBatch
import backtype.storm.transactional.TransactionAttempt; //导入依赖的package包/类
@Override
public MemoryTransactionalSpoutMeta emitPartitionBatch(TransactionAttempt tx, BatchOutputCollector collector, int partition, MemoryTransactionalSpoutMeta lastPartitionMeta) {
if(!Boolean.FALSE.equals(getDisabledStatuses().get(partition))) {
int index;
if(lastPartitionMeta==null) {
index = 0;
} else {
index = lastPartitionMeta.index + lastPartitionMeta.amt;
}
List<List<Object>> queue = getQueues().get(partition);
int total = queue.size();
int left = total - index;
int toTake = Math.min(left, _takeAmt);
MemoryTransactionalSpoutMeta ret = new MemoryTransactionalSpoutMeta(index, toTake);
for(int i=ret.index; i < ret.index + ret.amt; i++) {
List<Object> toEmit = new ArrayList<Object>(queue.get(i));
toEmit.add(0, tx);
collector.emit(toEmit);
}
if(toTake==0) {
// this is a pretty hacky way to determine when all the partitions have been committed
// wait until we've emitted max-spout-pending empty partitions for the partition
int curr = Utils.get(_emptyPartitions, partition, 0) + 1;
_emptyPartitions.put(partition, curr);
if(curr > _maxSpoutPending) {
getFinishedStatuses().put(partition, true);
}
}
return ret;
} else {
return null;
}
}
开发者ID:metamx,项目名称:incubator-storm,代码行数:35,代码来源:OpaqueMemoryTransactionalSpout.java
注:本文中的backtype.storm.transactional.TransactionAttempt类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。 |
请发表评论