µÚÒ»·¶ÎÄÍø - רҵÎÄÕ·¶ÀýÎĵµ×ÊÁÏ·ÖÏíÆ½Ì¨

Ïȵç´óÊý¾Ýƽ̨²Ù×÷ÊÖ²á-iandian-bigdata-v2.1

À´Ô´£ºÓû§·ÖÏí ʱ¼ä£º2025/12/17 5:19:23 ±¾ÎÄÓÉloading ·ÖÏí ÏÂÔØÕâÆªÎĵµÊÖ»ú°æ
˵Ã÷£ºÎÄÕÂÄÚÈݽö¹©Ô¤ÀÀ£¬²¿·ÖÄÚÈÝ¿ÉÄܲ»È«£¬ÐèÒªÍêÕûÎĵµ»òÕßÐèÒª¸´ÖÆÄÚÈÝ£¬ÇëÏÂÔØwordºóʹÓá£ÏÂÔØwordÓÐÎÊÌâÇëÌí¼Ó΢ÐźÅ:xxxxxxx»òQQ£ºxxxxxx ´¦Àí£¨¾¡¿ÉÄܸøÄúÌṩÍêÕûÎĵµ£©£¬¸ÐлÄúµÄÖ§³ÖÓëÁ½⡣

hive> create table choice(userid int£¬classname string) row format delimited fields terminated by '\\t';

×¢Ò⣺'\\t'Ï൱ÓÚÒ»¸ötab¼üÅÌ¡£ ÏÔʾ¸Õ²Å´´½¨µÄÊý¾Ý±í: hive> show tables; 2. µ¼ÈëÊý¾Ý

½¨±íºó£¬¿ÉÒÔ´Ó±¾µØÎļþϵͳ»ò HDFS Öе¼ÈëÊý¾ÝÎļþ£¬µ¼ÈëÊý¾ÝÑùÀýÈçÏÂ: ÄÚÈÝÈçÏÂ(Êý¾ÝÖ®¼äÓÃtab¼ü¸ô¿ª)£º 1 xiapi 2 xiaoxue 3 qingqing

ÄÚÈÝÈçÏÂ(Êý¾ÝÖ®¼äÓÃtab¼ü¸ô¿ª)£º jack math sam china lucy english

ÄÚÈÝÈçÏÂ(Êý¾ÝÖ®¼äÓÃtab¼ü¸ô¿ª)£º 1 math 1 china 1 english 2 china 2 english 3 english

Ê×ÏÈÔÚ±¾µØ¡°/home/hadoop/ziliao¡±Ï°´ÕÕÉÏÃæ½¨Á¢Èý¸öÎļþ£¬ ²¢Ìí¼ÓÈçÉϵÄÄÚÈÝÐÅÏ¢¡£

3. °´ÕÕÏÂÃæµ¼ÈëÊý¾Ý¡£

hive> load data local inpath '/home/hadoop/ziliao/' overwrite into table userinfo;

hive> load data local inpath '/home/hadoop/ziliao/' overwrite into table classinfo;

hive> load data local inpath '/home/hadoop/ziliao/' overwrite into table choice;

²éѯ±íÊý¾Ý

hive> select * from userinfo; hive> select * from classinfo; hive> select * from choice; 4. ·ÖÇø

´´½¨·ÖÇø

hive> create table ptest(userid int) partitioned by (name string) row format delimited fields terminated by '\\t';

×¼±¸µ¼ÈëÊý¾Ý

ÄÚÈÝÈçÏÂ(Êý¾ÝÖ®¼äÓÃtab¼ü¸ô¿ª)£º 1 µ¼ÈëÊý¾Ý

hive> load data local inpath '/home/hadoop/ziliao/' overwrite into table ptest partition (name='xiapi');

²é¿´·ÖÇø

hive> dfs -ls /user/hive/warehouse/ptest/name=xiapi; ²éѯ·ÖÇø

hive> select * from ptest where name='xiapi'; ÏÔʾ·ÖÇø

hive> show partitions ptest;

¶Ô·ÖÇø²åÈëÊý¾Ý(ÿ´Î¶¼»á¸²¸ÇµôÔ­À´µÄÊý¾Ý):

hive> insert overwrite table ptest partition(name='xiapi') select id from userinfo where name='xiapi';

ɾ³ý·ÖÇø

hive> alter table ptest drop partition (name='xiapi') 5. Ͱ

¿ÉÒ԰ѱí»ò·ÖÇø×éÖ¯³ÉͰ£¬ ͰÊǰ´Ðзֿª×éÖ¯ÌØ¶¨×ֶΣ¬ ÿ¸öͰ¶ÔÓ¦Ò»¸ö reduce ²Ù×÷¡£ÔÚ½¨Á¢Í°Ö®Ç°£¬ ÐèÒªÉèÖá°ÊôÐÔΪ true£¬ ʹ Hive Äܹ»Ê¶±ðͰ¡£ÔÚ±íÖзÖͰµÄ²Ù×÷ÈçÏÂ:

hive> set set create table btest2(id int£¬ name string) clustered by(id) into 3 buckets row format delimited fields terminated by '\\t';

ÏòͰÖвåÈëÊý¾Ý£¬ ÕâÀï°´ÕÕÓû§ id ·ÖÁËÈý¸öͰ£¬ ÔÚ²åÈëÊý¾Ýʱ¶ÔÓ¦Èý¸ö reduce ²Ù×÷£¬Êä³öÈý¸öÎļþ¡£

hive> insert overwrite table btest2 select * from userinfo; ²é¿´Êý¾Ý²Ö¿âϵÄͰĿ¼£¬Èý¸öͰ¶ÔÓ¦Èý¸öĿ¼¡£ hive> dfs -ls /user/hive/warehouse/btest2;

Hive ʹÓöԷÖͰËùÓõÄÖµ½øÐÐ hash£¬²¢Óà hash ½á¹û³ýÒÔͰµÄ¸öÊý×öÈ¡ÓàÔËËãµÄ·½Ê½À´·ÖͰ£¬±£Ö¤ÁËÿ¸öͰÖж¼ÓÐÊý¾Ý£¬µ«Ã¿¸öͰÖеÄÊý¾ÝÌõÊý²»Ò»¶¨ÏàµÈ£¬ÈçÏÂËùʾ¡£

hive>dfs -cat /user/hive/warehouse/btest2/*0_0; hive>dfs -cat /user/hive/warehouse/btest2/*1_0; hive>dfs -cat /user/hive/warehouse/btest2/*2_0;

·ÖͰ¿ÉÒÔ»ñµÃ±È·ÖÇø¸ü¸ßµÄ²éѯЧÂÊ£¬Í¬Ê±·ÖͰҲ±ãÓÚ¶ÔÈ«²¿Êý¾Ý½øÐвÉÑù´¦Àí¡£ÏÂÃæÊǶÔͰȡÑùµÄ²Ù×÷¡£

hive>select * from btest2 tablesample(bucket 1 out of 3 on id); 6. ¶à±í²åÈë

¶à±í²åÈëÖ¸µÄÊÇÔÚͬһÌõÓï¾äÖУ¬ °Ñ¶ÁÈ¡µÄͬһ·ÝÔªÊý¾Ý²åÈëµ½²»Í¬µÄ±íÖС£Ö»ÐèҪɨÃèÒ»±éÔªÊý¾Ý¼´¿ÉÍê³ÉËùÓбíµÄ²åÈë²Ù×÷£¬ ЧÂʺܸߡ£¶à±í²Ù×÷ʾÀýÈçÏ¡£

hive> create table mutill as select id£¬name from userinfo; #ÓÐÊý¾Ý hive> create table mutil2 like mutill; #ÎÞÊý¾Ý£¬Ö»Óбí½á¹¹ hive> from userinfo insert overwrite table mutill

select id£¬name insert overwrite table mutil2 select count(distinct id)£¬name group by name; 7. Á¬½Ó

Á¬½ÓÊǽ«Á½¸ö±íÖÐÔÚ¹²Í¬Êý¾ÝÏîÉÏÏ໥ƥÅäµÄÄÇЩÐкϲ¢ÆðÀ´£¬ HiveQL µÄÁ¬½Ó·ÖΪÄÚÁ¬½Ó¡¢×óÏòÍâÁ¬½Ó¡¢ÓÒÏòÍâÁ¬½Ó¡¢È«ÍâÁ¬½ÓºÍ°ëÁ¬½Ó 5 ÖÖ¡£

a. ÄÚÁ¬½Ó(µÈÖµÁ¬½Ó)

ÄÚÁ¬½ÓʹÓñȽÏÔËËã·û¸ù¾Ýÿ¸ö±í¹²ÓеÄÁеÄֵƥÅäÁ½¸ö±íÖеÄÐС£ ÀýÈ磬 ¼ìË÷userinfoºÍchoice±íÖбêʶºÅÏàͬµÄËùÓÐÐС£

hive> select userinfo.*£¬ choice.* from userinfo join choice on=; b. ×óÁ¬½Ó

×óÁ¬½ÓµÄ½á¹û¼¯°üÀ¨¡°LEFT OUTER¡±×Ó¾äÖÐÖ¸¶¨µÄ×ó±íµÄËùÓÐÐУ¬ ¶ø²»½ö½öÊÇÁ¬½ÓÁÐËùÆ¥ÅäµÄÐС£Èç¹û×ó±íµÄijÐÐÔÚÓÒ±íÖÐûÓÐÆ¥ÅäÐУ¬ ÔòÔÚÏà¹ØÁªµÄ½á¹û¼¯ÖÐÓÒ±íµÄËùÓÐÑ¡ÔñÁоùΪ¿ÕÖµ¡£

hive> select userinfo.*£¬ choice.* from userinfo left outer join choice on=; c. ÓÒÁ¬½Ó

ÓÒÁ¬½ÓÊÇ×óÏòÍâÁ¬½ÓµÄ·´ÏòÁ¬½Ó£¬½«·µ»ØÓÒ±íµÄËùÓÐÐС£Èç¹ûÓÒ±íµÄijÐÐÔÚ×ó±íÖÐûÓÐÆ¥ÅäÐУ¬Ôò½«Îª×ó±í·µ»Ø¿ÕÖµ¡£

hive> select userinfo.*£¬ choice.* from userinfo right outer join choice on=; d. È«Á¬½Ó

È«Á¬½Ó·µ»Ø×ó±íºÍÓÒ±íÖеÄËùÓÐÐС£µ±Ä³ÐÐÔÚÁíÒ»±íÖÐûÓÐÆ¥ÅäÐÐʱ£¬ÔòÁíÒ»¸ö±íµÄÑ¡ÔñÁбí°üº¬¿ÕÖµ¡£Èç¹û±íÖ®¼äÓÐÆ¥ÅäÐУ¬ÔòÕû¸ö½á¹û¼¯°üº¬»ù±íµÄÊý¾ÝÖµ¡£

hive> select userinfo.*£¬ choice.* from userinfo full outer join choice on=; e. °ëÁ¬½Ó

°ëÁ¬½ÓÊÇ Hive ËùÌØÓеģ¬ Hive ²»Ö§³Ö IN ²Ù×÷£¬µ«ÊÇÓµÓÐÌæ´úµÄ·½°¸; left semi join£¬ ³ÆÎª°ëÁ¬½Ó£¬ ÐèҪעÒâµÄÊÇÁ¬½ÓµÄ±í²»ÄÜÔÚ²éѯµÄÁÐÖУ¬Ö»ÄܳöÏÖÔÚ on ×Ó¾äÖС£

hive> select userinfo.* from userinfo left semi join choice on =; 8. ×Ó²éѯ

±ê×¼ SQL µÄ×Ó²éѯ֧³ÖǶÌ×µÄ select ×Ӿ䣬HiveQL ¶Ô×Ó²éѯµÄÖ§³ÖºÜÓÐÏÞ£¬Ö»ÄÜÔÚfrom Òýµ¼µÄ×Ó¾äÖгöÏÖ×Ó²éѯ¡£ÈçÏÂÓï¾äÔÚ from ×Ó¾äÖÐǶÌ×ÁËÒ»¸ö×Ó²éѯ(ʵÏÖÁ˶Խ̿Î×î¶àµÄÀÏʦµÄ²éѯ)¡£

hive>select teacher£¬MAX(class_num) from (select teacher,count(classname) as class_num from classinfo group by teacher) subq group by teacher; 9. ÊÓͼ²Ù×÷

Ŀǰ£¬Ö»ÓÐ Ö®ºóµÄ°æ±¾²ÅÖ§³ÖÊÓͼ¡£

Hive Ö»Ö§³ÖÂß¼­ÊÓͼ£¬ ²¢²»Ö§³ÖÎïÀíÊÓͼ£¬ ½¨Á¢ÊÓͼ¿ÉÒÔÔÚ MySQL ÔªÊý¾Ý¿âÖп´µ½´´½¨µÄÊÓͼ±í£¬ µ«ÊÇÔÚ Hive µÄÊý¾Ý²Ö¿âĿ¼ÏÂûÓÐÏàÓ¦µÄÊÓͼ±íĿ¼¡£

µ±Ò»¸ö²éѯÒýÓÃÒ»¸öÊÓͼʱ£¬ ¿ÉÒÔÆÀ¹ÀÊÓͼµÄ¶¨Ò岢ΪÏÂÒ»²½²éѯÌṩ¼Ç¼¼¯ºÏ¡£ÕâÊÇÒ»ÖÖ¸ÅÄîµÄÃèÊö£¬ ʵ¼ÊÉÏ£¬ ×÷Ϊ²éѯÓÅ»¯µÄÒ»²¿·Ö£¬ Hive ¿ÉÒÔ½«ÊÓͼµÄ¶¨ÒåÓë²éѯµÄ¶¨Òå½áºÏÆðÀ´£¬ÀýÈç´Ó²éѯµ½ÊÓͼËùʹÓõĹýÂËÆ÷¡£

ÔÚÊÓͼ´´½¨µÄͬʱȷ¶¨ÊÓͼµÄ¼Ü¹¹£¬Èç¹ûËæºóÔٸıä»ù±¾±í(ÈçÌí¼ÓÒ»ÁÐ)½«²»»áÔÚÊÓͼµÄ¼Ü¹¹ÖÐÌåÏÖ¡£Èç¹û»ù±¾±í±»É¾³ý»òÒÔ²»¼æÈݵķ½Ê½±»Ð޸ģ¬Ôò¸ÃÊÓͼµÄ²éѯ½«±»ÎÞЧ¡£

ÊÓͼÊÇÖ»¶ÁµÄ£¬²»ÄÜÓÃÓÚ LOAD/INSERT/ALTER¡£

ÊÓͼ¿ÉÄܰüº¬ ORDER BY ºÍ LIMIT ×Ӿ䣬Èç¹ûÒ»¸öÒýÓÃÁËÊÓͼµÄ²éѯҲ°üº¬ÕâЩ×Ӿ䣬ÄÇôÔÚÖ´ÐÐÕâЩ×Ó¾äʱÊ×ÏÈÒª²é¿´ÊÓͼÓï¾ä£¬È»ºó·µ»Ø½á¹û°´ÕÕÊÓͼÖеÄÓï¾äÖ´ÐС£

ÒÔÏÂÊÇ´´½¨ÊÓͼµÄÀý×Ó:

hive> create view teacher_classsum as select teacher, count(classname) from classinfo group by teacher;

ɾ³ýÊÓͼ£º

hive>drop view teacher_classnum; 10. º¯Êý

´´½¨º¯Êý

hive> create temporary function function_name as class_name

¸ÃÓï¾ä´´½¨Ò»¸öÓÉÀàÃûʵÏֵĺ¯Êý¡£ÔÚ Hive ÖÐÓû§¿ÉÒÔʹÓà Hive Àà·¾¶ÖеÄÈκÎÀ࣬Óû§Í¨¹ýÖ´ÐÐ add files Óï¾ä½«º¯ÊýÀàÌí¼Óµ½Àà·¾¶£¬²¢ÇҿɳÖÐøÊ¹Óøú¯Êý½øÐвÙ

ËÑË÷¸ü¶à¹ØÓÚ£º Ïȵç´óÊý¾Ýƽ̨²Ù×÷ÊÖ²á-iandian-bigdata-v2 µÄÎĵµ
Ïȵç´óÊý¾Ýƽ̨²Ù×÷ÊÖ²á-iandian-bigdata-v2.1.doc ½«±¾ÎĵÄWordÎĵµÏÂÔØµ½µçÄÔ£¬·½±ã¸´ÖÆ¡¢±à¼­¡¢ÊղغʹòÓ¡
±¾ÎÄÁ´½Ó£ºhttps://www.diyifanwen.net/c9reqv47be06m3qp9xkwe9ersa9ps1u00x9f_9.html£¨×ªÔØÇë×¢Ã÷ÎÄÕÂÀ´Ô´£©

Ïà¹ØÍÆ¼ö£º

ÈÈÃÅÍÆ¼ö
Copyright © 2012-2023 µÚÒ»·¶ÎÄÍø °æÈ¨ËùÓÐ ÃâÔðÉùÃ÷ | ÁªÏµÎÒÃÇ
ÉùÃ÷ :±¾ÍøÕ¾×ðÖØ²¢±£»¤ÖªÊ¶²úȨ£¬¸ù¾Ý¡¶ÐÅÏ¢ÍøÂç´«²¥È¨±£»¤ÌõÀý¡·£¬Èç¹ûÎÒÃÇ×ªÔØµÄ×÷Æ·ÇÖ·¸ÁËÄúµÄȨÀû,ÇëÔÚÒ»¸öÔÂÄÚ֪ͨÎÒÃÇ£¬ÎÒÃǻἰʱɾ³ý¡£
¿Í·þQQ£ºxxxxxx ÓÊÏ䣺xxxxxx@qq.com
ÓåICP±¸2023013149ºÅ
Top