Saturday, 15 February 2014

hadoop - Hive Insert works fine from Hive CLI but fails from terminal -


I currently have the following status:

I have a firefighter that creates two tables and Then one fills one with the data of the other.

My script looks to something like this:

  hive -e "external table table 1 ... roE format limited areas' \ t 'LOCATION' / User / input / '' hive -e '' External table table 2 ... divided by ... "hive -e" data as (SELECT date, ...) data table overwritten by table 2 partition ( Part_date) ... date " 

and then some other shellscript that selects some data from table 2. I use shellscript Because there are some arguments that I can select on table2 before I can be applied.

The script runs without error, tables are created, There is data in this Table 1 but the table2 ends empty. For some reasons this works when I use a very small test dataset, but as soon as the dataset is large (> 1 GB) table2 is empty.

If I'm too much All the commands work manually from the HIVVs CLI and the required data is in table2 .

Why does this happen, and how can I fix it?

To get data in your shell script command table2 Serial should be executed gradually.

Try this in your shell script:

  Hive-A "your first query" & amp; Amp; Hive-A "Your Second Query" & amp; Amp; Hive-A "Your Third Query"  

It should execute your grandson questions one by one. (Waits for 1 to finish the second query and waits for both the first and the second to end the third question)


No comments:

Post a Comment