HDP 2.5 설치

• OS : ubuntu 14.04
• 구성 : VM 서버 8대
• 기본 환경
  • 클라우드 VM 서버 :
    10.161.64.x hadoop-dev-1
    10.161.64.x hadoop-dev-2
    10.161.64.x hadoop-dev-3
    10.161.64.x hadoop-dev-4
    10.161.64.x hadoop-dev-5
    10.161.64.x hadoop-dev-6
    10.161.64.x hadoop-dev-7
    10.161.64.x hadoop-dev-8
     
• 방화벽 오픈 관련
  Source ——– Taget ——– Port
  운영서버 ——- any ———- 80, 443, 9418
  클라우드VM —– any ———- 80, 443, 9418, 123(NTP)
   
• hadoop-dev-1 에 SSH 접속

• root 권한 얻기

  # sudo -i

• root 홈에서 ssh key 생성

  # ssh-keygen
  # cat .ssh/id_rsa.pub >> authorized_keys

• 공개키(id_rsa.pub)를 모든 클라우드 VM에 복사 ※ sshpass가 없으면 설치 : apt-get install sshpass

  • 파일 복사

    # sshpass -f pass.txt scp -o StrictHostKeyChecking=no .ssh/id_rsa.pub ai_admin@hadoop-dev-4:id_rsa.pub

  • 파일 .ssh/경로에 복사 후 authorized_keys에 공개키 추가

    # cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

  • 참고 사항

    # sshpass -f pass.txt ssh ai_admin@hadoop-dev-1 “sudo cp id_rsa.pub ~/.ssh/;sudo bash -c "cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys"”

• 호스트 등록 - 아래 목록 추가

  # vi /etc/hosts
  10.161.64.1 hadoop-dev-1
  10.161.64.2 hadoop-dev-2
  10.161.64.3 hadoop-dev-3
  10.161.64.4 hadoop-dev-4
  10.161.64.5 hadoop-dev-5
  10.161.64.6 hadoop-dev-6
  10.161.64.7 hadoop-dev-7
  10.161.64.8 hadoop-dev-8

• NTP 동기화 http://webdir.tistory.com/208

  # sudo apt-get install ntp
  # sudo vi /etc/ntp.conf
   
  server 222.122.134.177
  server 222.122.169.177
   
  # sudo service ntp restart
  # sudo ntpq -p
   

• repository 등록

  # apt-key adv –recv-keys –keyserver keyserver.ubuntu.com B9733A7A07513CAD
   
  만약 아래와 같이 에러 발생시
  Executing: gpg –ignore-time-conflict –no-options –no-default-keyring –homedir /tmp/tmp.E527MxgV39 –no-auto-check-trustdb –trust-model always –keyring /etc/apt/trusted.gpg –primary-keyring /etc/apt/trusted.gpg –keyring /etc/apt/trusted.gpg.d/openjdk-r-ppa.gpg –keyring /etc/apt/trusted.gpg.d/webupd8team-java.gpg –recv-keys –keyserver keyserver.ubuntu.com B9733A7A07513CAD
  gpg: requesting key 07513CAD from hkp server keyserver.ubuntu.com
  gpgkeys: key B9733A7A07513CAD can’t be retrieved
  gpg: no valid OpenPGP data found.
  gpg: Total number processed: 0
  정보 : http://stackoverflow.com/questions/13112400/gpgkeys-key-7f0ceb10-not-found-on-keyserver-response-while-try-to-install-mon

  # apt-key adv –recv-keys –keyserver hkp://keyserver.ubuntu.com:80 B9733A7A07513CAD
  # apt-get update
  # apt-cache showpkg ambari-server
  # apt-cache showpkg ambari-agent
  # apt-cache showpkg ambari-metrics-assembly
  # apt-get install ambari-server

• ambari 설치

  # ambari-server setup
  자바 버전만 [1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8을 선택하고 나머지는 기본 옵션 선택

  에러발생)
  ERROR: -: /usr/bin/psql: /usr/bin/perl: bad interpreter: Permission denied -> chmod 755 /usr/bin/perl
  ERROR: Exiting with exit code 126.
  REASON: Running database init script failed. Exiting.
   

  # ambari-server start

• 웹을 통해 install ( http://10.107.92.56:8080/ )
   
Tip
• 네트워크 확인 : # telnet ip port
• 네트워크 상태 확인 : # netstat -an
• disable Transparent Huge Pages : https://docs.mongodb.com/v3.2/tutorial/transparent-huge-pages/https://access.redhat.com/solutions/46111

• /etc/profile.d/logging.sh: line 7: ifconfig: command not found-su: /usr/bin/find: Permission denied

  # sudo chmod 755 /usr/bin/find
  # sudo chmod 755 /bin/ps
  # sudo chmod 755 /bin/which
   

• ImportError: cannot import name _common

  # sudo python /usr/lib/python2.6/site-packages/resource_monitoring/psutil/build.py
  # cd /usr/lib/python2.6/site-packages/resource_monitoring/psutil
  # sudo python setup.py install
   

• ai_admin으로 hive 명령 permission Error
  hadoop fs -chmod 777 /user
  ai_admin으로 접속 -> 만약 /user 폴더에 ai_admin 폴더가 생겼다면, 다시 755로 변경하면 될 것임
   

• sqoop mssql driver : https://www.cloudera.com/documentation/enterprise/5-5-x/topics/cdh_ig_jdbc_driver_install.html

  # curl -L ‘http://download.microsoft.com/download/0/2/A/02AAE597-3865-456C-AE7F-613F99F850A8/sqljdbc_4.0.2206.100_enu.tar.gz’ | tar xz
  # sudo cp sqljdbc_4.0/enu/sqljdbc4.jar /usr/hdp/2.5.0.0-1245/sqoop/lib/
   

• 운영서버에서 암바리 서버를 설치 후, 각 노드에 agent 를 배포한 뒤에, 다른 노드를 암바리 서버로 하여 재설치하는 경우
  
  
  /etc/ambari-agent/conf/ambari-agent.ini 파일에  hostname에 이전 운영서버 호스트 이름이 등록되어 있어서, 문제가 발생하였음

• sqoop Error :
  java.lang.ClassNotFoundException: org.apache.atlas.sqoop.hook.SqoopHook
  at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
  at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
  at java.lang.Class.forName0(Native Method)
  at java.lang.Class.forName(Class.java:264)
  at org.apache.sqoop.mapreduce.PublishJobData.publishJobData(PublishJobData.java:46)
  at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:284)
  at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:692)
  at org.apache.sqoop.manager.SQLServerManager.importTable(SQLServerManager.java:163)
  at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:507)
  at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:615)
  at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
  at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
  at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
  at org.apache.sqoop.Sqoop.runTool(Sqoop.java:225)
  at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
  at org.apache.sqoop.Sqoop.main(Sqoop.java:243)
   

  ai_admin@hadoop-dev-1:/usr/hdp/2.5.0.0-1245/sqoop/lib$ ln -s /usr/hdp/current/atlas-server/hook/sqoop/atlas-plugin-classloader-0.7.0.2.5.0.0-1245.jar atlas-plugin-classloader-0.7.0.2.5.0.0-1245.jar
  ai_admin@hadoop-dev-1:/usr/hdp/2.5.0.0-1245/sqoop/lib$ ln -s /usr/hdp/current/atlas-server/hook/sqoop/sqoop-bridge-shim-0.7.0.2.5.0.0-1245.jar sqoop-bridge-shim-0.7.0.2.5.0.0-1245.jar
   

• MsSQL 서버에서 데이터 가져오기

  # sqoop import –connect ‘jdbc:sqlserver://10.100.150.128:31051;username=bireader;password=패스워드;database=mBI_SA’ –table TB_ParsingGameLog_20161027 –split-by regdatetime -m 3 –hcatalog-database default –hcatalog-table puzzlepet_orc_20161027 –create-hcatalog-table –hcatalog-storage-stanza “stored as orcfile” – –schema puzzlepet

• Ambari Alert Message ( /var/log/ambari-server/ambari-alerts.log
• 메시지 : [CRITICAL] [HARD] [AMBARI_METRICS] [ams_metrics_collector_process] (Metrics Collector Process) Connection failed: [Errno 111] Connection refused to hadoop-dev-1:6188
• 위의 메시지가 일정 주기마다 발생하였다. 이를 해결하기 위해 1번 노드에 설치된 ambari-metric을 16번 노드로 이동시키니 해결되었음. 예측컨데, 모든 서버 관련 프로세스가 1번 노드에만 설치되었기 때문에 발생한 문제로 생각됨.