Recent questions tagged Hadoop

0 votes

528 views

1 answer

hadoop - Oozie Workflow failed due to error JA017

I am using the version of Apache Oozie 4.3.0 along with Hadoop 2.7.3 I have developed a ... NodeManager 3470 SecondaryNameNode 4190 Bootstrap See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

486 views

1 answer

hadoop - Writing MApreduce code for counting number of records

I want to write a mapreduce code for counting number of records in given CSV file.I am not getting what ... this can anyone suggest something? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

650 views

1 answer

hadoop - Hive: Merging Configuration Settings not working

On Hive 2.2.0, I am filling an orc table from another source table of size 1.34 GB, using the query INSERT ... files at the end of the job. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

822 views

1 answer

hadoop - handling newline character in hive

I have created a table in hive as Create table(id int, Description String) My data looks something as ... newline before loading into hive. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

557 views

1 answer

hadoop - Sqoop: Could not load mysql driver exception

I Installed Sqoop in my local machine. Following are the config information. Bash.bashrc: export HADOOP_HOME=/home ... be the possible problem. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

590 views

1 answer

hadoop - Partition Hive table by existing field?

Can I partition a Hive table upon insert by an existing field? I have a 10 GB file with a date field ... FROM staging_event evt; Thanks! Travis See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.0k views

1 answer

hadoop - Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses-submiting job2remoteClustr

I recently upgraded my cluster from Apache Hadoop1.0 to CDH4.4.0. I have a weblogic server in another ... fine. Appreciate your timely help! See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

626 views

1 answer

hadoop - Is there any Conditional IF like operator in Apache PIG?

Actually I am writing PIG Script and want to execute some set of statements if one of the condition is satisfied. I ... can I do this? Thanks. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

820 views

1 answer

hadoop - Insert data into Hbase using Hive (JSON file)

I have already created a table in hbase using hive: hive> CREATE TABLE hbase_table_emp(id int, name string, role ... testemp; please Help ! :) See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

671 views

1 answer

hadoop - Migrate hive table to Google BigQuery

I am trying to design a sort of data pipeline to migrate my Hive tables into BigQuery. Hive is running ... creating the BigQuery table twice. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

601 views

1 answer

hadoop - Apache Pig permissions issue

I'm attempting to get Apache Pig up and running on my Hadoop cluster, and am encountering a permissions problem. ... what might be going on? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

677 views

1 answer

hadoop - AWS Glue issue with double quote and commas

I have this CSV file: reference,address V7T452F4H9,"12410 W 62TH ST, AA D" The following options are being used in ... How do I fix this issue? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

584 views

1 answer

hadoop - How to put files to specific node?

Is it possible to tell HDFS where to store particular files? Use case I've just loaded batch #1 of files into ... does it make sense at all? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

514 views

1 answer

hadoop - How does Hive choose the number of reducers for a job?

Several places say the default # of reducers in a Hadoop job is 1. You can use the mapred.reduce.tasks symbol ... .reduce.tasks=<number> ... See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

565 views

1 answer

hadoop - How to specify AWS Access Key ID and Secret Access Key as part of a amazon s3n URL

I am passing input and output folders as parameters to mapreduce word count program from webpage. Getting ... properties (respectively). See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

689 views

1 answer

hadoop - Failed to set permissions of path: mp

Failed to set permissions of path: mphadoop-MayPaynemapredstagingMayPayne2016979439.staging to 0700 I'm getting this error ... all in advance See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

661 views

1 answer

hadoop - how does hdfs choose a datanode to store

As the title indicates, when a client requests to write a file to the hdfs, how does the HDFS or name node ... a certain datanode as he likes? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

734 views

1 answer

hadoop - Why can't hive recognize alias named in select part?

Here's the scenario: When I invoke hql as follows, it tells me that it cannot find alias for u1. hive> select ... EOF at 'where' near 'user' See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

635 views

1 answer

hadoop - HBase Error : zookeeper.znode.parent mismatch

i am trying to learn Hadoop and i'v reached HBase section in Hadoop Definitive Guide. i tried to start ... detailed' is there anything wrong? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

741 views

1 answer

hadoop - could only be replicated to 0 nodes instead of minReplication (=1). There are 4 datanode(s) running and no node(s) are excluded in this operation

I don't know how to fix this error: Vertex failed, vertexName=initialmap, vertexId=vertex_1449805139484_0001_1_00, diagnostics ... 's the case? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

753 views

1 answer

hadoop - how to include external jar file using PIG

When I run a mapreduce job using hadoop command, I use -libjars to setup my jar to the cache and the ... do something like this in PIG? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

715 views

1 answer

hadoop - how to include external jar file using PIG

When I run a mapreduce job using hadoop command, I use -libjars to setup my jar to the cache and the ... do something like this in PIG? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

585 views

1 answer

hadoop - Unable to start a node manager on master

I am setting up a Hadoop YARN cluster and I am using a machine as both a master and a slave. When I ... TCP server1.mydomain.com:ampify (LISTEN See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

550 views

1 answer

hadoop - When are files "splittable"?

When I'm using spark, I sometimes run into one huge file in a HIVE table, and I sometimes am trying to ... of the two cases described above? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

592 views

1 answer

hadoop - Oozie + Sqoop: JDBC Driver Jar Location

I have a 6 node cloudera based hadoop cluster and I'm trying to connect to an oracle database from a sqoop ... driver jar? Thanks in advance! See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

520 views

1 answer

hadoop - How to pull data in the Map/Reduce functions?

According to the Hadoop : The Definitive Guide. The new API supports both a push and a pull style of ... API or example for the same. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

749 views

1 answer

hadoop - Why does all columns get created as string when I use OpenCSVSerde in Hive?

I am trying to create a table using the OpenCSVSerde and some integer and date columns. But the columns get ... change of datatype to String. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

534 views

1 answer

hadoop - Unable to Create Table in HIVE reading a CSV from HDFS

I have issues while creating a table in Hive by reading the .csv file from HDFS. The Query is below: CREATE EXTERNAL ... a .sql file as a job See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

Categories

Just Browsing Browsing

Most popular tags