hadoop - Optimised Hive query with JOIN , having million records -

-

मेरे पास 2 टेबल हैं - 

  bpm_agent_data - 40 मिलियन रिकॉर्ड, 5 कॉलम bpm_loan_data - 20 लाख रिकॉर्ड, 5 कॉलम

अब मैंने हाइव में एक प्रश्न चलाया - 

  चुनें count (bpm_agent_data.AgentID), count (bpm_loan_data.LoanNumber) bpm_agent_data से JOIN bpm_loan_data जहां bpm_loan_data.id = bpm_agent_data.id;

जो पूरा करने के लिए लंबा समय ले रहा है। HIVE में क्वेरी लिखने का आदर्श तरीका क्या होना चाहिए ताकि Reducer को इतना समय न लेना चाहिए।

उपरोक्त क्वेरी के लिए समाधान मिला, जहां पर 

  काउंट (bpm_agent_data.AgentID), गिनती (bpm_loan_data.LoanNumber) से bpm_agent_data पर जोड़ें bpm_loan_data पर (bpm_loan_data.id = bpm_agent_data.id );

Comments

Post a Comment

Popular posts from this blog

python - Writing Greek in matplotlib labels, titles -

-
Image
I'm trying to write some text for labels, shape titles etc. with my plots, but no benefit till now Has not happened . I do not want to print specific characters (I know how to use special characters), I want to write the entire text in Greek (maybe Unicode and text ' ?). , suggesting that I would like to mention that for some reason I can not get matplotlib to cooperate with Tex (using IPython notebook from Anaconda in Ubuntu 14.10) so that there really is an option Will not be the first. I tried loading the Arial font and it loads successfully but again I get square blocks instead of characters. I import matplotlib.font_manager FM PROP = FM Font Properties (fname = '/ usr / share / fonts / truetype / msttcorefonts / Arial.ttf') And then to display the string I u'Î? ? I ?? Î¿Ï ?? ÎμÎ »second ?? μΠ± i ?? I ± i ?? I ?? Used ο किया? ? Î¿Î¼Î¿Î¯Ï ?? I ?? I · I ?? ' . Ariel is considered to be fully presented to Greek and I have used it many times ...
Read more

c# - LINQ to Entities does not recognize the method 'Int32 IndexOf(System.String, System.StringComparison)' method -

-
I have performed a linq query using Entityframework as below GroupMaster getGroup = Null; GetGroup = DataContext.Groups.FirstOrDefault (item = & gt; keyword.IndexOf (item.Keywords, StringComparison.OrdinalIgnoreCase)> = 0 & amp; amp; item.ISNAmd) I found an exception like the method when executing it LINQ units do not recognize the 'Inter 32 index of (system string, system. String compromise)' method, and This method can not be translated to Store expression. Includes () method is sensitive by default, then I need to convert to minimize. Is there any method in which a string match match is examined and whether there is any method to solve the index question of the method? You actually have four options Databases globally Change the colon of it can be done in many ways, a simple Google search should reveal to them. Change the mix of individual tables or columns. Use a stored procedure and specify the call statement on your query Perform...
Read more

Pygame memory leak with transform.flip -

-
Today I came across an interesting issue which, by writing a small side scroller, instead of a sprite sheet, I was planning to use because I lacked a good "packer" for making phantom sheets (and honestly with patience and passion as it is only a test project). Anyway, I showed the animation running on the right of 8 PNG (with transparency). I considered it a good idea to prepare the flipped images to run on the left in the consultant, and the frame is not framed when the game is running (Flip actually consumes a lot of CPU like I Was measured). > To get access to the loop, I get the memory leak self.move_l = [# 8 PNG images with transparency to the right] For the self in IMG. Move_l: self.move_r.append (pygame.transform.flip (img, true, false) error traceback: traceback (most recent Call final): The file "D: \ Python 34 \ _Projects \ efg \ efg.py", in line 196, in the & lt; module & gt; main = master program ((1024, 680)) "D: \ Python ...
Read more