Skip to main content

Hive temporary table auto deletion

Hive temporary table auto deletion


Steps for Temporary Hive table:

I have checked that there is no such table (not temporary) already exists.

hive -e 'use myschema; show tables "tmp";'
--no rows returned

Then I run :

$hive -e 'use myschema; create temporary table tmp (id int);'
OK

Check there is no table:

hive -e 'use myschema; show tables "tmp";'
--no rows returned - it works correctly

Create not temporary table:

hive -e 'use myschema; create table tmp (id int);'
--ok
Now there is persistent table:

hive -e 'use myschema; show tables "tmp";'
OK
tmp
Time taken: 0.081 seconds, Fetched: 1 row(s)
Try to create the same temporary table:

hive -e 'use myschema; create temporary table tmp (id int);'
OK
Persistent table remains in the schema. Temporary table was dropped successfully and temporary table was isolated inside session, not visible to other sessions.

Comments

Post a Comment

Popular posts from this blog

Minus query in HIVE

It does not appear that HQL supports the MINUS operator What you want to do can be done with a LEFT JOIN or NOT EXISTS: SELECT x FROM abc LEFT JOIN bcd ON abc.x = bcd.x WHERE bcd.x IS NULL EDIT: Per comments below, NOT EXISTS is not supported. SELECT x FROM abc WHERE NOT EXISTS (SELECT x FROM bcd)
1 comment
Read more

INCREMENTAL UPDATES IN HIVE TABLES

  In this blog I will focus on  Incremental load OR updating the exiting records and inserting new record  and  dynamic partition table loading . In BI world delta load/incremental load is very common activity.       We can implement the same on HIVE table using below steps. In a relation database we will perform updating the exiting records using unique index/primary key, but in Hive we have to do it in different way.  There are many ways to do it, let us see how we can do it using below steps. I am assuming that you have data in HDFS which will be updating hourly/daily from your sqoop/flume/kafka. Design consideration:   I am assuming that good temp space is maintained according to your data volume, so that we not facing temp space issue during the process. I am assuming that you have ingestion logic set to in your code for full load and incremental loads and data is available in HDFS.  Partition column value s...
3 comments
Read more

Remove or Replace unwanted simple like space, $ and comma (,) from hive data

hive -e 'select regexp_replace("$60,825.48","\\$|\,","");' OK 60825.48 Time taken: 3.912 seconds, Fetched: 1 row(s) hive -e 'select regexp_replace("$60,825.48","[^0-9.]","");' 60825.48 Time taken: 2.249 seconds, Fetched: 1 row(s) substring from a string using the TRANSLATE function to replace the substring with the empty string. For your query it would become this: drop table vp_hiphop; create table vp_hiphop as select userid, ntext,        regexp_replace(regexp_replace(ntext, 'hip hop', 'hiphop'), 'rock music', 'rockmusic') as ntext1 from  vp_nlp_protext_males; drop table vp_hiphop; create table vp_hiphop as select  userid, ntext,         translate(ntext, ' ', '') as ntext1 from  vp_nlp_protext_males;
Post a Comment
Read more