Recent questions tagged Hadoop

0 votes

606 views

1 answer

hadoop - Hive inserting values to an array complex type column

I am unable to append data to tables that contain an array column using insert into statements; the data type ... into columns using jdbc ? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

977 views

1 answer

hadoop - How to cross join unnest a JSON array in Presto

Given a table that contains a column of JSON like this: {"payload":[{"type":"b","value":"9"}, {" ... expand all array elements and select them? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

514 views

1 answer

hadoop - How to change sqoop metastore?

I am using sqoop 1.4.2 version. I am trying to change the sqoop metastore from default hsqldb to mysql. I ... than hsql db? Please suggest. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.0k views

1 answer

hadoop - How do I get millisecond precision in hive?

The documentation says that timestamps support the following conversion: ?Floating point numeric types: Interpreted ... additional information. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

572 views

1 answer

hadoop mapreduce: java.lang.UnsatisfiedLinkError: org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z

I am trying to write a snappy block compressed sequence file from a map-reduce job. I am using hadoop 2.0.0- ... to me what I am doing wrong? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

747 views

1 answer

hadoop - copying directory from local system to hdfs java code

I'm having a problem trying to copy a directory from my local system to HDFS using java code. I'm able to ... me with that? Thanks in advance. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

549 views

1 answer

hadoop - Hive doesn't support in, exists. How do I write the following query?

I have two tables A and B that both have a column id. I wish to obtain ids from A that are not present in B. ... is not equal to any id in A. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

988 views

1 answer

hadoop - Incorrect configuration: namenode address dfs.namenode.rpc-address is not configured

I am getting this error when I try and boot up a DataNode. From what I have read, the RPC paramters are ... ? EDIT: Just use Cloudera manager. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

948 views

1 answer

hadoop - MapReduce job hangs, waiting for AM container to be allocated

I tried to run simple word count as MapReduce job. Everything works fine when run locally (all work done on ... not in Yarn ResourceManager ) See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

544 views

1 answer

hadoop - manipulating iterator in mapreduce

I am trying to find the sum of any given points using hadoop, The issue I am having is on getting all values from ... is the right way to do it. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

542 views

1 answer

hadoop - YARN Resourcemanager not connecting to nodemanager

thanks in advance for any help I am running the following versions: Hadoop 2.2 zookeeper 3.4.5 Hbase 0.96 Hive ... -1:8033</value> </property> See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

526 views

1 answer

hadoop - How to keep YARN's log files?

Suddenly, my YARN cluster has stopped working, everything I submit fails with "Exit code 1". I want to track ... YARN to keep these log files? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

732 views

1 answer

hadoop - What exactly Non DFS Used means?

This is what I saw on Web UI recently Configured Capacity : 232.5 GB DFS Used : 112.44 GB Non DFS Used : ... . What happend? Did I made mistake? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

515 views

1 answer

hadoop - How do you use s3a with spark 2.1.0 on aws us-east-2?

Background I have been working on getting a flexible setup for myself to use spark on aws with docker swarm mode ... being used for some reason. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

522 views

1 answer

hadoop - Hive is not showing tables

I am new to Hadoop and Hive world. I have a strange problem. When I was working on hive prompt. I have ... Can you please help me on this? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.0k views

1 answer

hadoop - The root scratch dir: /tmp/hive on HDFS should be writable. Current permissions are: -wx------

I have changed permission using hdfs command. Still it showing same error. The root scratch dir: /tmp/hive on HDFS ... are: rwx------ See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

612 views

1 answer

hadoop - Apache Spark error : Could not connect to akka.tcp://sparkMaster@

This is our first steps using big data stuff like apache spark and hadoop. We have a installed Cloudera CDH 5. ... run the application I got the See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

670 views

1 answer

hadoop - Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database

I am trying to run SparkSQL : val sqlContext = new org.apache.spark.sql.hive.HiveContext(sc) But the ... the error shows as derby execption See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

671 views

1 answer

hadoop - Hive Create Multi small files for each insert in HDFS

following is already been achieved Kafka Producer pulling data from twitter using Spark Streaming. Kafka Consumer ingesting ... in one file? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

556 views

1 answer

hadoop - On what basis mapreduce framework decides whether to launch a combiner or not

As per definition "The Combiner may be called 0, 1, or many times on each key between the mapper and ... many times cobiner will be launched. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

650 views

1 answer

hadoop - Connecting to Hive using Beeline

I am trying to connect to hive installed in my machine through Beeline client. when I give the 'beeline' ... ) in some configuration file? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

472 views

1 answer

hadoop - $HADOOP_HOME is deprecated

I started a hadoop cluster. I get this warning message: $HADOOP_HOME is deprecated I already add export ... , It shows again. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.0k views

1 answer

hadoop - HIVE nested ARRAY in MAP data type

I have HIVE table structured like this:- Create table test_stg(employee_id INT, name STRING, abu ARRAY <String>, ... array. Thanks in advance. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

659 views

1 answer

hadoop - How to flatten a group into a single tuple in Pig?

From this: (1, {(1,2), (1,3), (1,4)} ) (2, {(2,5), (2,6), (2,7)} ) ...How ... over many rows AND manipulate internal groups at the same time. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

861 views

1 answer

hadoop - Computing median in map reduce

Can someone example the computation of median/quantiles in map reduce? My understanding of Datafu's median is that ... the final task. Thanks See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

647 views

1 answer

hadoop - Pig vs Hive vs Native Map Reduce

I've basic understanding on what Pig, Hive abstractions are. But I don't have a clear idea on the scenarios ... Hive but in native map reduce? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

745 views

1 answer

hadoop - How to count lines in a file on hdfs command?

I have a file on HDFS that I want to know how many lines are. (testfile) In linux, I can do: wc -l < ... I have to use -text instead of -cat See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.1k views

1 answer

hadoop - Alter hive table add or drop column

I have orc table in hive I want to drop column from this table ALTER TABLE table_name drop col_name; but I am getting ... , I am using hive 0.14 See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

Categories

Just Browsing Browsing

Most popular tags