Recent questions tagged MapReduce

0 votes

474 views

1 answer

mapreduce - Write to multiple tables in HBASE

I have a situation here where I need to write to two of the hbase tables say table1,table 2. Whenever ... should not include co-processors. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

399 views

1 answer

mapreduce - hadoop converting to and breaking ARC format

I am trying to parse data from commoncrawl.org using hadoop streaming. I set up a local hadoop to ... eventually run on AWS ElasticMapReduce. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

397 views

1 answer

mapreduce - How to handle id generation on a hadoop cluster?

I am building a dictionary on a hadoop cluster and need to generate a numeric id for each token. How should I do it? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

375 views

1 answer

mapreduce - How to change the structure of MongoDB's map-reduce results?

When I'm running Map-Reduce on a Mongo database, I usually get results similar to the following: { _id: < ... withing a map or reduce function? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

549 views

1 answer

mapreduce - Wordcount program is stuck in hadoop-2.3.0

I installed hadoop-2.3.0 and tried to run wordcount example But it starts the job and sits idle hadoop@ubuntu:~ ... hadoop-2.3.0 Please suggest. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

346 views

1 answer

mapreduce - Configure Map Side join for multiple mappers in Hadoop Map/Reduce

I have a question about configuring Map/Side inner join for multiple mappers in Hadoop. Suppose I have two ... pairs of files simultaneously? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

429 views

1 answer

mapreduce - "merge" view collation into useful output in CouchDB

When doing a "join" in CouchDB, you can use view collation to group the records together. For example, having two ... is how exactly to do this? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

503 views

1 answer

mapreduce - "merge" view collation into useful output in CouchDB

When doing a "join" in CouchDB, you can use view collation to group the records together. For example, having two ... is how exactly to do this? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

442 views

1 answer

mapreduce - Implementation of an ArrayWritable for a custom Hadoop type

How do I define an ArrayWritable for a custom Hadoop type ? I am trying to implement an inverted index in ... values in the Reduce method See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

487 views

1 answer

mapreduce - Hadoop namenode : Single point of failure

The Namenode in the Hadoop architecture is a single point of failure. How do people who have large Hadoop clusters ... the primary one fails ? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

449 views

1 answer

mapreduce - Fast way to find duplicates on indexed column in mongodb

I have a collection of md5 in mongodb. I'd like to find all duplicates. The md5 column is indexed. Do you ... { out.duplicates.insert(obj) }); See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

527 views

1 answer

mapreduce - What are SUCCESS and part-r-00000 files in hadoop

Although I use Hadoop frequently on my Ubuntu machine I have never thought about SUCCESS and part-r-00000 files ... just a randomly defined? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

420 views

1 answer

mapreduce - How does Hadoop decide how many nodes will perform the Map and Reduce tasks?

I'm new to hadoop and I'm trying to understand it. Im talking about hadoop 2. When I have an input file ... nodes will do the reduce tasks? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

333 views

1 answer

mapreduce - Hadoop: No Such Method Exception

I have written a MapReduce program, code is below: import java.io.IOException; import java.util.Iterator; import org ... I rectify it? Thanks! See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

351 views

1 answer

mapreduce - What is Keyword Context in Hadoop programming world?

What exactly is this keyword Context in Hadoop MapReduce world in new API terms? Its extensively used to write ... for your time and help. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

350 views

1 answer

mapreduce - Hadoop: how to access (many) photo images to be processed by map/reduce?

I have 10M+ photos saved on the local file system. Now I want to go through each of them to analyze the ... terms of the above question, Thanks! See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

459 views

1 answer

mapreduce - Output a list from a Hadoop Map Reduce job using custom writable

I'm trying to create a simple map reduce job by changing the wordcount example given by hadoop. I'm trying ... job.waitForCompletion(true); } } See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

428 views

1 answer

mapreduce - About Hadoop/HDFS file splitting

Want to just confirm on following. Please verify if this is correct: 1. As per my understanding when we copy ... available to a single mapper. See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

872 views

1 answer

mapreduce - Hadoop WordCount example stuck at map 100% reduce 0%

[hadoop-1.0.2] → hadoop jar hadoop-examples-1.0.2.jar wordcount /user/abhinav/input /user/abhinav/output ... . What could be causing this? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

358 views

1 answer

mapreduce - Map-Reduce performance in MongoDb 2.2, 2.4, and 2.6

I've found this discussion: MongoDB: Terrible MapReduce Performance. Basically it says try to avoid Mongo's MR ... to effectively use indexes? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

498 views

1 answer

mapreduce - Map Reduce with F# agents

After playing with F# agents I tried to do a map reduce using them. The basic structure I use is: map supervisor ... ) mp.Status() mp.Stop() See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

503 views

1 answer

mapreduce - hadoop: difference between 0 reducer and identity reducer?

I am just trying to confirm my understanding of difference between 0 reducer and identity reducer. 0 reducer means ... will still take place? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

667 views

1 answer

mapreduce - Split size vs Block size in Hadoop

What is relationship between split size and block size in Hadoop? As I read in this, split size must be n ... between split size and block size? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

502 views

1 answer

mapreduce - Hadoop input split size vs block size

I am going through hadoop definitive guide, where it clearly explains about input splits. It goes like Input splits ... lesser than block size? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

685 views

1 answer

mapreduce - What is the use of grouping comparator in hadoop map reduce

I would like to know why grouping comparator is used in secondary sort of mapreduce. According to the definitive ... it. Thanks in advance See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

486 views

1 answer

mapreduce - How does Hadoop perform input splits?

This is a conceptual question involving Hadoop/HDFS. Lets say you have a file containing 1 billion lines. And ... val parameters? Thanks, Deepak See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

543 views

1 answer

mapreduce - Find all duplicate documents in a MongoDB collection by a key field

Suppose I have a collection with some set of documents. something like this. { "_id" : ObjectId(" ... "bar" appears 3 times. See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

545 views

1 answer

mapreduce - Remove Duplicates from MongoDB

hi I have a ~5 million documents in mongodb (replication) each document 43 fields. how to remove duplicate document ... can solve the problem ? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

Categories

Just Browsing Browsing

Most popular tags