hadoop mapreduce 串行执行,比较慢,请问可能什么原因
hive>insert overwrite table archive_seller_by_geo_per_day partition(partDate, shard) select sellerTokenID, partDate archiveDate, countryCode, country, countryState, countryCity, count(*) pv, count(distinct ip) uv, partDate, shard from page_visit where partDate >= 20140105 and partDate <= 20140107 group by sellerTokenID, countryCode, country, countryState, countryCity, partDate, shard having sellerTokenID>0;
2014-01-06 10:48:31,590 null map = 0%, reduce = 0%
2014-01-06 10:48:46,353 null map = 1%, reduce = 0%
2014-01-06 10:48:48,775 null map = 2%, reduce = 0%
2014-01-06 10:49:04,329 null map = 5%, reduce = 0%
2014-01-06 10:49:22,239 null map = 7%, reduce = 0%
2014-01-06 10:49:25,585 null map = 100%, reduce = 0%
2014-01-06 10:49:27,814 null map = 12%, reduce = 0%
2014-01-06 10:49:46,299 null map = 15%, reduce = 0%
2014-01-06 10:50:06,873 null map = 19%, reduce = 0%
2014-01-06 10:50:11,868 null map = 100%, reduce = 0%
2014-01-06 10:50:14,524 null map = 22%, reduce = 0%
2014-01-06 10:50:30,130 null map = 25%, reduce = 0%
2014-01-06 10:50:33,458 null map = 26%, reduce = 0%
2014-01-06 10:50:44,565 null map = 28%, reduce = 0%
2014-01-06 10:50:51,427 null map = 30%, reduce = 0%
2014-01-06 10:50:53,719 null map = 100%, reduce = 0%
2014-01-06 10:50:56,403 null map = 34%, reduce = 0%
2014-01-06 10:51:15,112 null map = 37%, reduce = 0%
2014-01-06 10:51:29,410 null map = 41%, reduce = 0%
2014-01-06 10:51:32,758 null map = 100%, reduce = 0%
2014-01-06 10:51:35,993 null map = 45%, reduce = 0%
2014-01-06 10:51:50,467 null map = 48%, reduce = 0%
2014-01-06 10:52:06,274 null map = 52%, reduce = 0%
2014-01-06 10:52:07,383 null map = 100%, reduce = 0%
2014-01-06 10:52:10,282 null map = 56%, reduce = 0%
2014-01-06 10:52:25,854 null map = 59%, reduce = 0%
2014-01-06 10:52:40,382 null map = 63%, reduce = 0%
2014-01-06 10:52:41,565 null map = 100%, reduce = 0%
2014-01-06 10:52:44,892 null map = 67%, reduce = 0%
2014-01-06 10:52:55,913 null map = 70%, reduce = 0%
2014-01-06 10:53:11,398 null map = 74%, reduce = 0%
2014-01-06 10:53:13,589 null map = 100%, reduce = 0%
2014-01-06 10:53:15,911 null map = 78%, reduce = 0%
2014-01-06 10:53:28,068 null map = 82%, reduce = 0%
2014-01-06 10:53:36,848 null map = 84%, reduce = 0%
2014-01-06 10:53:43,340 null map = 85%, reduce = 0%
2014-01-06 10:53:44,579 null map = 100%, reduce = 0%
2014-01-06 10:53:46,846 null map = 89%, reduce = 0%
2014-01-06 10:53:56,057 null map = 96%, reduce = 0%
2014-01-06 10:53:57,162 null map = 100%, reduce = 0%
2014-01-06 10:54:03,018 null map = 100%, reduce = 67%
2014-01-06 10:54:06,450 null map = 100%, reduce = 68%
2014-01-06 10:54:08,777 null map = 100%, reduce = 69%
2014-01-06 10:54:14,706 null map = 100%, reduce = 70%
2014-01-06 10:54:17,933 null map = 100%, reduce = 71%
2014-01-06 10:54:21,429 null map = 100%, reduce = 72%
2014-01-06 10:54:24,682 null map = 100%, reduce = 73%
2014-01-06 10:54:26,845 null map = 100%, reduce = 74%
2014-01-06 10:54:30,375 null map = 100%, reduce = 75%
2014-01-06 10:54:33,781 null map = 100%, reduce = 76%
2014-01-06 10:54:36,054 null map = 100%, reduce = 77%
2014-01-06 10:54:39,347 null map = 100%, reduce = 78%
2014-01-06 10:54:42,764 null map = 100%, reduce = 79%
2014-01-06 10:54:44,912 null map = 100%, reduce = 80%
2014-01-06 10:54:48,182 null map = 100%, reduce = 81%
2014-01-06 10:54:51,553 null map = 100%, reduce = 82%
2014-01-06 10:54:53,814 null map = 100%, reduce = 83%
2014-01-06 10:54:57,005 null map = 100%, reduce = 84%
2014-01-06 10:55:00,303 null map = 100%, reduce = 85%
2014-01-06 10:55:06,918 null map = 100%, reduce = 86%
2014-01-06 10:55:09,084 null map = 100%, reduce = 87%
2014-01-06 10:55:12,400 null map = 100%, reduce = 88%
2014-01-06 10:55:15,788 null map = 100%, reduce = 89%
2014-01-06 10:55:17,930 null map = 100%, reduce = 90%
2014-01-06 10:55:21,452 null map = 100%, reduce = 91%
2014-01-06 10:55:24,815 null map = 100%, reduce = 92%
2014-01-06 10:55:27,028 null map = 100%, reduce = 93%
2014-01-06 10:55:30,222 null map = 100%, reduce = 94%
2014-01-06 10:55:33,535 null map = 100%, reduce = 95%
2014-01-06 10:55:36,771 null map = 100%, reduce = 96%
2014-01-06 10:55:39,249 null map = 100%, reduce = 97%
2014-01-06 10:55:42,651 null map = 100%, reduce = 98%
2014-01-06 10:55:45,888 null map = 100%, reduce = 99%
2014-01-06 10:55:50,289 null map = 100%, reduce = 100%
Time taken: 499.09 seconds
日志没有错误,结果也执行正常,但速度太慢,请问可能是什么原因导致的呢?谢谢~~
当麻当麻-
11 years, 10 months ago