menu
more_vert
Hue for Apache Hadoop

作者:dayu

2. Hue architecture

Hue applications run in a Web browser and require no client installation.The following figure illustrates how Hue works. Hue Server is a "container" web application that sits in between hadoop installation and the browser. It hosts all the Hue web applications and communicates with various servers that interface withhadoopcomponents.

技术分享

3.Install and configureHue on Hadoop

Hue consists of a web service that runs on a special node in the hadoop cluster.At here,I willuse existing master node as the Hue Server.

3.1 Technical Details

  • Distribution: Apache hadoop HDFS
  • Cluster Manager:Resource manager
  • Environment: Ali cloud server
  • Operating System: Ubuntu 14.04 LTS

3.2 Features confirmed to work in partial or complete fashion

  • File Browser (HDFS access through WebHdfs or HttpFS)
  • Hive/Beeswax (Beeswax uses the Hive client libraries)
  • HBase Cluster Browser (Requires Thrift 1 service)
  • job Browser (Job information access through hue-plugins)

3.3 Hue Dependencies

Hue employs some Python modules which use native code and requires certain development libraries be installed on the system. To install from the tarball, following components must be installed:

  • sudo apt-get install -y ant
  • sudo apt-get install -y gcc g++
  • sudo apt-get install -y libkrb5-dev libmysqlclient-dev
  • sudo apt-get install -y libssl-dev libsasl2-dev libsasl2-modules-gssapi-mit
  • sudo apt-get install -y libsqlite3-dev
  • sudo apt-get install -y libtidy-0.99-0 libxml2-dev libxslt-dev
  • sudo apt-get install -y maven
  • sudo apt-get install -y libldap2-dev
  • sudo apt-get install -y python-dev python-simplejson python-setuptools

3.4 Install and configurehue

(1) Download Hue 3.9 release tarball from below link.

http://gethue.com/hue-3-9-with-all-its-improvements-is-out/

(2) Unpackage the tarball to the Ubuntuoptdirectory.

技术分享

(3) Make sure all the dependencies have been installed and then start the build process.

技术分享

By default, Hue installs to ‘/usr/local/hue’ inthe masternode’s local filesystem.

技术分享

(4) The Hue configuration file can be found at ‘/usr/local/hue/desktop/conf/hue.ini’ . Bellowarethe changes.

Desktop

技术分享

Hadoop

技术分享

Hbase

技术分享

Hive

技术分享

3.5 Start hue and browser the data.

(1) Start the Hue server using the ‘supervisor’ command.
技术分享
(2) Also need to start hive2 server for using Hive
技术分享
(3) Start Hbase server and thrift server for using Hbase.

技术分享

3.6 Browser thedata with Hue.

(1) Using hsql to select demo datafrom hive database

技术分享

(2) Create demo table by using Hbase browser

技术分享

(3) Check jobsinformation

技术分享

4.Tips for some common issues

(1) Sometimes the build process for the dependencies raises errors. So I suggest to manually install Ant and Maven.For these two tools, you can directly download corresponding release tarballs and then manually configure environment variable ANT_HOME and MAVEN_HOME and PATH.

(2) As installed, the Hue installation folders and file ownership will be set to the ‘root’ user. we‘d better to fix that so Hue can run correctly without root user permissions.

(3) For error message"creating build/temp.linux-x86_64-2.7/src gcc -pthread -fno-strict-aliasing -fwrapv -Wall -Wstrict-prototypes -fPIC -std=c99 -O3 -fomit-frame-pointer -Isrc/ -I/usr/include/ -I/home/huser/miniconda/include/python2.7 -c src/_fastmath.c -o build/temp.linux-x86_64-2.7/src/_fastmath.o src/_fastmath.c:36:18: fatal error: gmp.h: No such file or directory # include <gmp.h> ^ compilation terminated. error: command ‘gcc‘ failed with exit status 1..."

is because that gcc had not found "gmp.h". You need to assure that you have installed "libgmp3-dev" package and have gmp.h in path. Try to execute below command to install libgmp3-dev.

#sudo apt-get install libgmp3-dev

Hue for Apache Hadoop

原文地址:http://www.cnblogs.com/kinginme/p/7204976.html