보물창고/Big Data2013.05.06 23:30






Hortonworks sandbox 가상머신 내 링크

아파치 링크


Hortonworks 샌드박스에 있는 Ambari 관련 내용 번역 입니다







Apache Ambari - Making Hadoop easier to operate (아파치 Ambari - 하둡을 쉽게 작동 하도록 만들기)

아파츠 Ambari는 아파치 하둡 클러스터를 모니터링하고 효율적으로 관리하기 위해 100% 오픈소스 도구로 직관적인 세트를 제공합니다. Ambari가 작어을 단순화 하고 하둡의 복잡성을 숨깁니다. 하둡을 하나로 모은 데이터 플랫폼 같은 화면으로 나타냅니다

참여 노드가 수백 또는 수천인 경우 하둡 클러스터 구축 및 지속적인 관리는 특히 복잡한 작업이 될 수 있습니다. 

Ambari는 업데이트 및 하둡 서비스 수명 주기를 관리를 위해 단일 보기 제어 지점을 제공합니다. Ambari의 주요 기능중 일부는 다음과 같습니다.

마법사 기반 설치
하둡 서비스와 구성 요소의 세부 구성
metrics 수집 및 시스템 경고에 대한 Nagios를 위한 Ganglia
상세 Job 진단 및 문제 해결 도구
클러스터 히트 맵

당신이 클러스터를 설치할 때 Ambari는 첫번째 플레이로 온다. Ambari는 클러스터에 있는 하둡 패키지의 설치를 관리합니다. 사용자 인터페이스를 통해 당신은 NameNode에서, Job추적 및 datanodes로 클러스터의 각 노드에 역할을 할당 할 수 있습니다. 모든 패키지를 설치한 후 Ambari설치가 성공했는지 확인하기 위해 각 역할에 대한 테스트를 실행합니다

설치 후 Ambari는 클러스터를 관리하고 모니터링 하는데 필요한 두구를 제공합니다. 당신에게 클러스터의 상태를 표시하는 대시보드를 제공합니다. 관리 및 모니터링 작업을 하고 클러스터 리소스를 사용하는 방법을 이해 할 수 있습니다. 

-

Enabling Ambari in the Hortonworks Sandbox (Hortonworks 샌드 박스에서 Ambari 활성화)

root로 로그인



Hortonworks 샌드 박스에 Ambari를 구현하는 첫번째 단계를 root로 로그인 하는 것입니다. 당신의 가상머신은 다음과 같습니다

로그인 프롬프트를 얻으려면 당신은 Alt + F5 키를 입력해야 합니다. 정확한 키는 가상 플랫폼마다 다릅니다. 내 노트북 Virtual Box의 OS X는 FN-CNTRL-ALT F5를 입력합니다. 화면이 변경됩니다 그리고 로그인 하십시오




로그인 ID는 root이고 암호는 hadoop입니다


-

Running the script (스크립트 실행)

당신은 디렉토리 내용을 나열하는 ls 명령을 입력할 수 있습니다. Ambari를 활성화 하는데 사용되는 스크립트 "start_ambari.sh"를 볼 수 있습니다





"./start_ambari.sh 스크립트를 실행 합니다. 당신은 스크립트가 완료되면 서버를 시작하고 오류를 나타내고 프롬프트로 돌아갑니다

이제 가상 컴퓨터를 다시 시작하려면 "reboot"명령을 입력합니다


-


Modifying the hosts file on your computer (컴퓨터 호스트 파일을 수정)

Ambari 인터페이스의 링크 브라우저는 가상머신의 IP주소 이름으로 샌드박스를 연결 해야합니다
당신 컴퓨터에 있는 hosts파일에 다음 라인을 추가해야합니다

<샌드박스 IP 주소> sandbox

OS X 스스템에서 호스트 파일은 다음과 같이 표시됩니다




호스트 팡리 위치는 시스템에 따라 다릅니다. 리눅스 및 Mac OS X에 댛나 호스트 파일은 "/etc"에 있습니다. 윈도우의 경우 "C:\Windows\System32\Drivers\etc\hosts"입니다. 윈도우 8은 Windows Defender에서 hosts파일을 악성코드로 감지하므로 참조하십시오사용하기 위해 필요한 변경 사항들이 있습니다. IP주소는 세션마다 변경될 수 있으므로 이 단계를 다시 실행해야 할 수도 있습니다.


--


Performance tips and login information for Sandbox(성능 정보 및 샌드 박스의 로그인 정보)

가상 컴퓨터 RAM을 추가 - Ambari추가 서비스를 실행 활성화합니다. 당신의 샌드박스 가상 머신의 성능이 느릴것 같으면 당신은 RAM을 추가해 가상머신 설정을 수정할 수 있습니다. 당신은 변경하기전에 가상머신을 중간 해야 합니다. 여러분 시스템이 변경사항을 적용하기에 충분한 물리적 RRAM이 있는 경우에 우리는 2GB에서 3GB이상으로 올릴것을 권장 합니다. 가상머신에 할당된 RAM을 증가하는 방법에 대한 자세한 내용은 가상화 플랫폼에 대한 설명서를 참조하십시오.

가상머신 CPU코어를 추가 - 멀티코어 프로세서에서 실행중인 경우 샌드박스 가상 머신을 빠르게 하는 또 다른 방법은 가상 컴퓨터에 두 번째 코어를 할당하는 것입니다. 당신은 샌드 박스 가상 머신을 중지한 후에 가상 머신 환경에서 이러한 변경을 할 수 있습니다. 가상 컴퓨터에 두 번째 코어를 추가하는 방법에 대한 지침은 가상화 플랫폼에 대한 설명서를 참조하십시오. 

Nagios의 연결시에는 로그인과 암호를 요청합니다. 아이디는 nagiosadmin이고 암호는 admin입니다

-

Connecting to Ambari( Ambari 접속)

일단 당신이 Ambari를 활성화하고 사용할 샌드박스 가상머신을 재부팅합니다. 브라우저에서 개인 샌드박스 가상머신주소:8080 로 연결합니다. 다음 로그인 페이지가 표시됩니다.




기본 사용자 아이디 admin과 기본 비밀번호 admin을 입력합니다. 그런다음 Ambari대시 보드를 참조하십시오. 왼쪽의 대시 보드는 클러스터 서비스 상태를 보여줍니다. 오른쪽은 클러스터 메트릭 차트 입니다. 상당엔 페이지 추가 버튼이 있습니다.




-


Using Ambari to monitor the Sandbox(Ambari를 사용하여 샌드박스를 모니터링)

이 다음 섹션에서 우리는 작업을 싲가하고 우리가 Ambari를 사용하여 작업을 모니터링하는 방법을 볼 수 있습니다. 우리는 Ambari대시 보드 및 옆 샌드박스 사용자 인터페이스 측면을 모두 볼 수있도록 첫번째 단계는 추가 브라우저 창을 여는 겁니다.





이제 이 샌드 박스에서 작업을 실행하고 우리가 Ambari에서 모니터링 할 수 있는 방법을 살펴봅니다. 샌드 박스의 상단에 저장된 쿼리를 클립합니다.





첫 번째 쿼리에 옵션 상자를 드롭다운 선택하면 쿼리를 복제 할 수 있는 옵션을 볼 수 있습니다.




복제를 클릭하면 쿼리의 복사본을 얻을 것입니다. 하단에 녹색 실행 버튼을 클릭하면 쿼리가 실행 됩니다.




작업이 완료되면 당신은 Ambari 대시 보드의 맨 위에있는 작업 버튼을 클릭할 수 있고 당신의 작업 통계를 볼 수 있습니다.





샌드박스 가상머신에서 시간 변위에 대한 주의사항 - 가상 컴퓨터에 대한 제한 사항 중 하나는 가상화 소프트웨어 이외의 변경 사항에 대한 가상 시스템인지 모릅니다. 가상 컴퓨터를 일시 중단하거나 귀하의 컴퓨터의 내의 가상머신의 시계를 잘 모릅니다. 일시 중단 된 동안 가상 컴퓨터가 다시 시작되면 시간이 부정확 합니다. 당신은 작업 통계를 보면 타임스탬프에 자신을 매니페스트합니다.


-

Learning more about Ambari (Ambari에대해 더 배우고 싶으면)

당신은 사용자 문서에서 더 많은 Ambari를 탐색할 수 있습니다.

신고





Posted by 파란물
보물창고/Big Data2013.05.01 12:56











이번 포스팅은 HortonWorks에서 제공하는 hadoop 튜토리얼중 oozie를 설치하면서 생긴 문제점들을 정리했습니다

CentOS6 


HortonWorks 튜토리얼 앞부분에 있는 환경 설정이 안되어 있다면 아래와 같이 oozie설치 진행이 안될 수 있습니다

oozie 3.2.0 Document 필요한 부분 번역






이 문제를 해결 하려면 oozie 환경설정을 수정하고 Native library를 설치해야 합니다

oozie-env.sh 환경 설정은 제일 아래 보시면 java.library.path에 적혀 있는 설정을 /usr/local/lib 로 변경합니다

native library 설치는  http://www.solanara.net/solanara/tomcat#head_tomcat_inst_native 이 주소를 참고하시면 도움이 되실 겁니다
위 포스팅에서 아래 두 단계만 설치하시면 됩니다 아파치 톰캣은 oozie가 내부적으로 가지고 있습니다 (따로 실행 하는것도 가능합니다)
2. 라이브러리 : Apache Portable Runtime 
5. Tomcat - Native library

아래 에러 문제 때문에 권한 바꾸고 진행하기
INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /usr/lib/hadoop/lib/native/Linux-amd64-64
su root
chown -R hadoop:hadoop /usr/lib/hadoop/lib/native/Linux-amd64-64
chmod -R 755 /usr/lib/hadoop/lib/native/Linux-amd64-64








su root
yum install oozie extjs-2.2-1








yum install oozie oozie-client









2.x으로 명령해서 실행되지 않은 쿼리
/usr/lib/oozie/bin/oozie-setup.sh -hadoop 2.x /usr/lib/hadoop -extjs /usr/share/HDP-oozie/ext-2.2.zip







/usr/lib/oozie/bin/oozie-setup.sh -hadoop 0.20.2 /usr/lib/hadoop -extjs /usr/share/HDP-oozie/ext-2.2.zip










2.x
로 해서 실행되지 않은 쿼리
/usr/lib/oozie/bin/oozie-setup.sh -hadoop 2.x /usr/lib/hadoop -extjs /usr/share/HDP-oozie/ext-2.2.zip -jars /usr/lib/hadoop/lib/hadoop-lzo-0.5.0.jar






/usr/lib/oozie/bin/oozie-setup.sh -hadoop 0.20.2 /usr/lib/hadoop -extjs /usr/share/HDP-oozie/ext-2.2.zip -jars /usr/lib/hadoop/lib/hadoop-lzo-0.5.0.jar










/usr/lib/oozie/bin/oozie-setup.sh -hadoop 2.x /usr/lib/hadoop -extjs /usr/share/HDP-oozie/ext-2.2.zip -jars /usr/lib/hadoop/lib/hadoop-lzo-0.5.0.jar:/usr/share/java/mysql-connector-java.jar








각각의 파일 소유자와 권한을 수정해 줍니다

chown -R hadoop:hadoop /var/log/oozie
chmod -R 755 /var/log/oozie

chown -R hadoop:hadoop /etc/oozie/conf.dist/
chmod -R 755 /etc/oozie/conf.dist/





튜토리얼에서 제공하는 conf파일 oozie에서 oozie-site.xml 파일을 수정
Hortonworks튜토리얼에서는 mysql을 이용하게 설명되어있는데 default DB인 derby를 이용해서 진행합니다

<property>
<name>oozie.base.url</name>
<value>http://localhost:11000/oozie</value>
<description>Enter your Oozie server hostname.</description>
</property>


<property>

    <name>oozie.service.JPAService.jdbc.driver</name>

    <value>org.apache.derby.jdbc.EmbeddedDriver</value>

    <description>

        JDBC driver class.

    </description>

</property>


<property>

    <name>oozie.service.JPAService.jdbc.url</name>

    <value>jdbc:derby:/home/hadoop/oozie/${oozie.db.schema.name}-db;create=true</value>

    <description>

        JDBC URL.

    </description>

</property>

<property>

    <name>oozie.service.JPAService.jdbc.username</name>

    <value>hadoop</value>

    <description>

        DB user name.

    </description>

</property>


<property>

    <name>oozie.service.JPAService.jdbc.password</name>

    <value>hadoop</value>

    <description>

        DB user password.

        IMPORTANT: if password is emtpy leave a 1 space string, the service trims the value,

                   if empty Configuration assumes it is NULL.

    </description>

</property>

hadoop/config/core-site.xml 아래에 이 설정 추가
  <property>
     <name>hadoop.proxyuser.hadoop.groups</name>      <value>users</value>      <description>Proxy group for Hadoop.</description>   </property>

  <property>      <name>hadoop.proxyuser.hadoop.hosts</name>      <value>*</value>      <description>Proxy host for Hadoop.</description>   </property>



oozie도 default conf에 파일이 있어서 수정한 conf파일을 default conf에 덮어 씌움








/home/hadoop/HDP~~~~~/conf~~~/oozie 파일을 수정

oozie-env.sh

Hortonworks 튜토리얼에서는 아래처럼 수정하라고 되어 있으나 실제 내용과는 달라서 아래 내용으로 수정함
<property>
<name>OOZIE_LOG_DIR</name>
<value>/var/log/oozie</value>
<description>Use value from $OOZIE_LOG_DIR </description>
</property>

<property>
<name>OOZIE_PID_DIR</name>
<value>/var/run/oozie</value>
<description>Use value from $OOZIE_PID_DIR </description>
</property>

<property>
<name>OOZIE_DATA_DIR</name>
<value>/var/db/oozie</value>
<description>Use value from $OOZIE_DATA_DIR </description>
</property>



실제 oozie-env.sh파일 내용이 이렇게 생겨서 위에 참고해서 수정함
# Oozie pid directory
#
export CATALINA_PID=/home/hadoop/id.pid

#Location of the data for oozie
export OOZIE_DATA=/home/hadoop/oozie


oozie-env.sh 환경설정 파일 가장 아래 부분에 설정
# set the libarary path to hadoop 64 bit native code dir
export JAVA_LIBRARY_PATH="/usr/local/lib"








su hadoop
/usr/lib/oozie/bin/oozie-start.sh
oozie-start.sh를 실행하니 나온 에러 해결
oozie-setup.sh 설정하는데 에러가 있었는데 확인 안된상태에서 진행 했더니 나오는 에러
2.x로 표시된 부분을 바꿔서 설정 했었어야함 0.20.2로 바꿔 명령 후 진행하니 문제없어짐

oozie-start.sh  oozie-stop.sh  
[hadoop@localhost ~]$ /usr/lib/oozie/bin/oozie-start.sh

Setting OOZIE_HOME:           /usr/lib/oozie
Sourcing:                     /usr/lib/oozie/bin/oozie-env.sh
  setting OOZIE_CONFIG=${OOZIE_CONFIG:-/etc/oozie/conf}
  setting OOZIE_DATA=${OOZIE_DATA:-/var/lib/oozie}
  setting OOZIE_LOG=${OOZIE_LOG:-/var/log/oozie}
  setting CATALINA_BASE=${CATALINA_BASE:-/var/lib/oozie/oozie-server}
  setting CATALINA_TMPDIR=${CATALINA_TMPDIR:-/var/tmp/oozie}
  setting CATALINA_PID=${CATALINA_PID:-/var/run/oozie/oozie.pid}
Using   OOZIE_CONFIG:         /etc/oozie/conf
Sourcing:                     /etc/oozie/conf/oozie-env.sh
  setting JAVA_HOME=/usr/java/default
  setting OOZIE_LOG=/var/log/oozie
  setting CATALINA_PID=/home/hadoop/id
  setting OOZIE_DATA=/home/hadoop/oozie
  setting JAVA_LIBRARY_PATH="/usr/lib/hadoop/lib/native/Linux-amd64-64"
Setting OOZIE_CONFIG_FILE:   oozie-site.xml
Using   OOZIE_DATA:           /home/hadoop/oozie
Using   OOZIE_LOG:            /var/log/oozie
Setting OOZIE_LOG4J_FILE:     oozie-log4j.properties
Setting OOZIE_LOG4J_RELOAD:  10
Setting OOZIE_HTTP_HOSTNAME: localhost
Setting OOZIE_HTTP_PORT:      11000
Setting OOZIE_ADMIN_PORT:      11001
Setting OOZIE_BASE_URL:       http://localhost:11000/oozie
Using   CATALINA_BASE:        /var/lib/oozie/oozie-server
Setting CATALINA_OUT:         /var/log/oozie/catalina.out
Using   CATALINA_PID:         /home/hadoop/id

Using   CATALINA_OPTS:         -Dderby.stream.error.file=/var/log/oozie/derby.log
Adding to CATALINA_OPTS:      -Doozie.home.dir=/usr/lib/oozie -Doozie.config.dir=/etc/oozie/conf -Doozie.log.dir=/var/log/oozie -Doozie.data.dir=/home/hadoop/oozie -Doozie.config.file=oozie-site.xml -Doozie.log4j.file=oozie-log4j.properties -Doozie.log4j.reload=10 -Doozie.http.hostname=localhost -Doozie.admin.port=11001 -Doozie.http.port=11000 -Doozie.base.url=http://localhost:11000/oozie -Djava.library.path=/usr/lib/hadoop/lib/native/Linux-amd64-64
WARN: Oozie WAR has not been set up at ''/var/lib/oozie/oozie-server/webapps'', doing default set up
  setting OOZIE_CONFIG=${OOZIE_CONFIG:-/etc/oozie/conf}
  setting OOZIE_DATA=${OOZIE_DATA:-/var/lib/oozie}
  setting OOZIE_LOG=${OOZIE_LOG:-/var/log/oozie}
  setting CATALINA_BASE=${CATALINA_BASE:-/var/lib/oozie/oozie-server}
  setting CATALINA_TMPDIR=${CATALINA_TMPDIR:-/var/tmp/oozie}
  setting CATALINA_PID=${CATALINA_PID:-/var/run/oozie/oozie.pid}
  setting JAVA_HOME=/usr/java/default
  setting OOZIE_LOG=/var/log/oozie
  setting CATALINA_PID=/home/hadoop/id
  setting OOZIE_DATA=/home/hadoop/oozie
  setting JAVA_LIBRARY_PATH="/usr/lib/hadoop/lib/native/Linux-amd64-64"

ERROR: Stop Oozie first 

[hadoop@localhost ~]$











su root
chown -R hadoop:hadoop /var/lib/oozie/
chmod -R 755 /var/lib/oozie/

mkdir /var/tmp/oozie
chown -R hadoop:hadoop /var/tmp/oozie/
chmod -R 755 /var/tmp/oozie/

su hadoop
/usr/lib/oozie/bin/oozie-start.sh



다른 에러가 나와서 해결중
/var/urn/oozie/catalina.2013-04-29.log 파일 내용

/var/lib/oozie/oozie-server/webapps/oozie/docs/oozie-default.xml 이 경로에 파일이 없음
webapps폴더 소유자가 oozie-oozie user로 되어 있어서인듯해 권한 수정 후 파일이 생성됨
su root
chown -R hadoop:hadoop /var/lib/oozie/oozie-server/webapps
chmod -R 755 /var/lib/oozie/oozie-server/webapps






Apr 29, 2013 3:54:37 AM org.apache.catalina.startup.Embedded initDirs
SEVERE: Cannot find specified temporary folder at /var/tmp/oozie
Apr 29, 2013 3:54:38 AM org.apache.catalina.core.AprLifecycleListener init
INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /usr/lib/hadoop/lib/native/Linux-amd64-64
Apr 29, 2013 3:54:39 AM org.apache.coyote.http11.Http11Protocol init
INFO: Initializing Coyote HTTP/1.1 on http-11000
Apr 29, 2013 3:54:39 AM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 1871 ms
Apr 29, 2013 3:54:39 AM org.apache.catalina.core.StandardService start
INFO: Starting service Catalina
Apr 29, 2013 3:54:39 AM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/6.0.32
Apr 29, 2013 3:54:39 AM org.apache.catalina.startup.HostConfig deployWAR
INFO: Deploying web application archive oozie.war
Apr 29, 2013 3:54:39 AM org.apache.catalina.startup.ContextConfig init
SEVERE: Exception fixing docBase for context [/oozie]
java.io.FileNotFoundException: /var/lib/oozie/oozie-server/webapps/oozie/docs/oozie-default.xml (No such file or directory)
    at java.io.FileOutputStream.open(Native Method)
    at java.io.FileOutputStream.<init>(FileOutputStream.java:194)
    at java.io.FileOutputStream.<init>(FileOutputStream.java:145)
    at org.apache.catalina.startup.ExpandWar.expand(ExpandWar.java:457)
    at org.apache.catalina.startup.ExpandWar.expand(ExpandWar.java:173)
    at org.apache.catalina.startup.ContextConfig.fixDocBase(ContextConfig.java:886)
    at org.apache.catalina.startup.ContextConfig.init(ContextConfig.java:1021)
    at org.apache.catalina.startup.ContextConfig.lifecycleEvent(ContextConfig.java:279)
    at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
    at org.apache.catalina.core.StandardContext.init(StandardContext.java:5700)
    at org.apache.catalina.core.StandardContext.start(StandardContext.java:4448)
    at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:799)
    at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:779)
    at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:601)
    at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:943)
    at org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:778)
    at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:504)
    at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1315)
    at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:324)
    at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
    at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1061)
    at org.apache.catalina.core.StandardHost.start(StandardHost.java:840)
    at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
    at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)
    at org.apache.catalina.core.StandardService.start(StandardService.java:525)
    at org.apache.catalina.core.StandardServer.start(StandardServer.java:754)
    at org.apache.catalina.startup.Catalina.start(Catalina.java:595)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
    at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)











위 폴더 권한 수정후 다른 에러 발생 확인 중
Existing PID file found during start.
Removing/clearing stale PID file.

/home/hadoop/id.pid파일을 삭제 하면 위에러는 나타나지 않음





[hadoop@localhost ~]$ /usr/lib/oozie/bin/oozie-start.sh

Setting OOZIE_HOME:           /usr/lib/oozie
Sourcing:                     /usr/lib/oozie/bin/oozie-env.sh
  setting OOZIE_CONFIG=${OOZIE_CONFIG:-/etc/oozie/conf}
  setting OOZIE_DATA=${OOZIE_DATA:-/var/lib/oozie}
  setting OOZIE_LOG=${OOZIE_LOG:-/var/log/oozie}
  setting CATALINA_BASE=${CATALINA_BASE:-/var/lib/oozie/oozie-server}
  setting CATALINA_TMPDIR=${CATALINA_TMPDIR:-/var/tmp/oozie}
  setting CATALINA_PID=${CATALINA_PID:-/var/run/oozie/oozie.pid}
Using   OOZIE_CONFIG:         /etc/oozie/conf
Sourcing:                     /etc/oozie/conf/oozie-env.sh
  setting JAVA_HOME=/usr/java/default
  setting OOZIE_LOG=/var/log/oozie
  setting CATALINA_PID=/home/hadoop/id.pid
  setting OOZIE_DATA=/home/hadoop/oozie
  setting JAVA_LIBRARY_PATH="/usr/lib/hadoop/lib/native/Linux-amd64-64"
Setting OOZIE_CONFIG_FILE:   oozie-site.xml
Using   OOZIE_DATA:           /home/hadoop/oozie
Using   OOZIE_LOG:            /var/log/oozie
Setting OOZIE_LOG4J_FILE:     oozie-log4j.properties
Setting OOZIE_LOG4J_RELOAD:  10
Setting OOZIE_HTTP_HOSTNAME: localhost
Setting OOZIE_HTTP_PORT:      11000
Setting OOZIE_ADMIN_PORT:      11001
Setting OOZIE_BASE_URL:       http://localhost:11000/oozie
Using   CATALINA_BASE:        /var/lib/oozie/oozie-server
Setting CATALINA_OUT:         /var/log/oozie/catalina.out
Using   CATALINA_PID:         /home/hadoop/id.pid

Using   CATALINA_OPTS:         -Dderby.stream.error.file=/var/log/oozie/derby.log
Adding to CATALINA_OPTS:      -Doozie.home.dir=/usr/lib/oozie -Doozie.config.dir=/etc/oozie/conf -Doozie.log.dir=/var/log/oozie -Doozie.data.dir=/home/hadoop/oozie -Doozie.config.file=oozie-site.xml -Doozie.log4j.file=oozie-log4j.properties -Doozie.log4j.reload=10 -Doozie.http.hostname=localhost -Doozie.admin.port=11001 -Doozie.http.port=11000 -Doozie.base.url=http://localhost:11000/oozie -Djava.library.path=/usr/lib/hadoop/lib/native/Linux-amd64-64

Using CATALINA_BASE:   /var/lib/oozie/oozie-server
Using CATALINA_HOME:   /usr/lib/oozie/oozie-server
Using CATALINA_TMPDIR: /var/tmp/oozie
Using JRE_HOME:         /usr/java/default
Using CLASSPATH:        /usr/lib/oozie/oozie-server/bin/bootstrap.jar
Using CATALINA_PID:     /home/hadoop/id.pid
Existing PID file found during start.
Removing/clearing stale PID file.

[hadoop@localhost ~]$











oozie 실행을 여러번 했더니 11000 포트에 대한 에러가 있어 해당 포트확인하고 pid확인해서 해결함
java.net.BindException: Address already in use <null>:11000

/usr/sbin/lsof -P -n -i tcp
해당하는 pid를 kill -9 로 강제 종료 시킴




Apr 29, 2013 4:21:45 AM org.apache.catalina.startup.Embedded initDirs
SEVERE: Cannot find specified temporary folder at /var/tmp/oozie
Apr 29, 2013 4:21:45 AM org.apache.catalina.core.AprLifecycleListener init
INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /usr/lib/hadoop/lib/native/Linux-amd64-64
Apr 29, 2013 4:21:46 AM org.apache.coyote.http11.Http11Protocol init
SEVERE: Error initializing endpoint
java.net.BindException: Address already in use <null>:11000
    at org.apache.tomcat.util.net.JIoEndpoint.init(JIoEndpoint.java:549)
    at org.apache.coyote.http11.Http11Protocol.init(Http11Protocol.java:176)
    at org.apache.catalina.connector.Connector.initialize(Connector.java:1022)
    at org.apache.catalina.core.StandardService.initialize(StandardService.java:703)
    at org.apache.catalina.core.StandardServer.initialize(StandardServer.java:838)
    at org.apache.catalina.startup.Catalina.load(Catalina.java:538)
    at org.apache.catalina.startup.Catalina.load(Catalina.java:562)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.catalina.startup.Bootstrap.load(Bootstrap.java:261)
    at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:413)
Caused by: java.net.BindException: Address already in use
    at java.net.PlainSocketImpl.socketBind(Native Method)
    at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:383)
    at java.net.ServerSocket.bind(ServerSocket.java:328)
    at java.net.ServerSocket.<init>(ServerSocket.java:194)
    at java.net.ServerSocket.<init>(ServerSocket.java:150)
    at org.apache.tomcat.util.net.DefaultServerSocketFactory.createSocket(DefaultServerSocketFactory.java:50)
    at org.apache.tomcat.util.net.JIoEndpoint.init(JIoEndpoint.java:538)
    ... 12 more
Apr 29, 2013 4:21:46 AM org.apache.catalina.core.StandardService initialize
SEVERE: Failed to initialize connector [Connector[HTTP/1.1-11000]]
LifecycleException:  Protocol handler initialization failed: java.net.BindException: Address already in use <null>:11000
    at org.apache.catalina.connector.Connector.initialize(Connector.java:1024)
    at org.apache.catalina.core.StandardService.initialize(StandardService.java:703)
    at org.apache.catalina.core.StandardServer.initialize(StandardServer.java:838)
    at org.apache.catalina.startup.Catalina.load(Catalina.java:538)
    at org.apache.catalina.startup.Catalina.load(Catalina.java:562)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.catalina.startup.Bootstrap.load(Bootstrap.java:261)
    at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:413)
Apr 29, 2013 4:21:46 AM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 849 ms
Apr 29, 2013 4:21:46 AM org.apache.catalina.core.StandardService start
INFO: Starting service Catalina
Apr 29, 2013 4:21:46 AM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/6.0.32
Apr 29, 2013 4:21:46 AM org.apache.catalina.startup.HostConfig deployWAR
INFO: Deploying web application archive oozie.war








/var/log/oozie/   (oozie 로그 쌓는 위치)
catalina.2013-04-29.log 파일
이렇게 출력 중인데 http://localhost:11000/oozie 페이지가 열리지 않아 확인
/log/oozie/catalina.out에 있는 에러 로그를 확인하면 더 정확하게 볼 수 있음

Apr 29, 2013 9:12:22 PM org.apache.catalina.startup.Embedded initDirs

SEVERE: Cannot find specified temporary folder at /var/tmp/oozie ## 해당 경로에 폴더를 만들어서 해결

Apr 29, 2013 9:12:22 PM org.apache.catalina.core.AprLifecycleListener init

INFO: Loaded APR based Apache Tomcat Native library 1.1.27.

Apr 29, 2013 9:12:22 PM org.apache.catalina.core.AprLifecycleListener init

INFO: APR capabilities: IPv6 [true], sendfile [true], accept filters [false], random [true].

Apr 29, 2013 9:12:22 PM org.apache.catalina.core.AprLifecycleListener lifecycleEvent

INFO: Failed to initialize the SSLEngine.

Apr 29, 2013 9:12:23 PM org.apache.coyote.http11.Http11AprProtocol init

INFO: Initializing Coyote HTTP/1.1 on http-11000

Apr 29, 2013 9:12:23 PM org.apache.catalina.startup.Catalina load

INFO: Initialization processed in 999 ms

Apr 29, 2013 9:12:23 PM org.apache.catalina.core.StandardService start

INFO: Starting service Catalina

Apr 29, 2013 9:12:23 PM org.apache.catalina.core.StandardEngine start

INFO: Starting Servlet Engine: Apache Tomcat/6.0.32

Apr 29, 2013 9:12:23 PM org.apache.catalina.startup.HostConfig deployWAR

INFO: Deploying web application archive oozie.war


Apr 29, 2013 10:27:37 PM org.apache.catalina.core.AprLifecycleListener init

INFO: Loaded APR based Apache Tomcat Native library 1.1.27.

Apr 29, 2013 10:27:37 PM org.apache.catalina.core.AprLifecycleListener init

INFO: APR capabilities: IPv6 [true], sendfile [true], accept filters [false], random [true].

Apr 29, 2013 10:27:38 PM org.apache.catalina.core.AprLifecycleListener lifecycleEvent

INFO: Failed to initialize the SSLEngine.

Apr 29, 2013 10:27:38 PM org.apache.coyote.http11.Http11AprProtocol init

INFO: Initializing Coyote HTTP/1.1 on http-11000

Apr 29, 2013 10:27:38 PM org.apache.catalina.startup.Catalina load

INFO: Initialization processed in 851 ms

Apr 29, 2013 10:27:38 PM org.apache.catalina.core.StandardService start

INFO: Starting service Catalina

Apr 29, 2013 10:27:38 PM org.apache.catalina.core.StandardEngine start

INFO: Starting Servlet Engine: Apache Tomcat/6.0.32

Apr 29, 2013 10:27:38 PM org.apache.catalina.startup.HostConfig deployDescriptor

INFO: Deploying configuration descriptor oozie.xml





/log/oozie/catalina.out 로그 내용

Apr 30, 2013 1:44:09 AM org.apache.catalina.core.AprLifecycleListener init

INFO: Loaded APR based Apache Tomcat Native library 1.1.27.

Apr 30, 2013 1:44:09 AM org.apache.catalina.core.AprLifecycleListener init

INFO: APR capabilities: IPv6 [true], sendfile [true], accept filters [false], random [true].

Apr 30, 2013 1:44:09 AM org.apache.catalina.core.AprLifecycleListener lifecycleEvent

INFO: Failed to initialize the SSLEngine.

Apr 30, 2013 1:44:09 AM org.apache.coyote.http11.Http11AprProtocol init

INFO: Initializing Coyote HTTP/1.1 on http-11000

Apr 30, 2013 1:44:09 AM org.apache.catalina.startup.Catalina load

INFO: Initialization processed in 861 ms

Apr 30, 2013 1:44:09 AM org.apache.catalina.core.StandardService start

INFO: Starting service Catalina

Apr 30, 2013 1:44:09 AM org.apache.catalina.core.StandardEngine start

INFO: Starting Servlet Engine: Apache Tomcat/6.0.32

Apr 30, 2013 1:44:09 AM org.apache.catalina.startup.HostConfig deployDescriptor

INFO: Deploying configuration descriptor oozie.xml


ERROR: Oozie could not be started


REASON: java.lang.NoClassDefFoundError: org/apache/commons/configuration/Configuration


Stacktrace:

-----------------------------------------------------------------

java.lang.NoClassDefFoundError: org/apache/commons/configuration/Configuration

   at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.<init>(DefaultMetricsSystem.java:37)

   at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.<clinit>(DefaultMetricsSystem.java:34)

   at org.apache.hadoop.security.UgiInstrumentation.create(UgiInstrumentation.java:51)

   at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:216)

   at org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:226)

   at org.apache.oozie.service.HadoopAccessorService.init(HadoopAccessorService.java:111)

   at org.apache.oozie.service.HadoopAccessorService.init(HadoopAccessorService.java:77)

   at org.apache.oozie.service.Services.setServiceInternal(Services.java:358)

   at org.apache.oozie.service.Services.setService(Services.java:344)

   at org.apache.oozie.service.Services.loadServices(Services.java:278)

   at org.apache.oozie.service.Services.init(Services.java:202)

   at org.apache.oozie.servlet.ServicesLoader.contextInitialized(ServicesLoader.java:39)

   at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4205)

   at org.apache.catalina.core.StandardContext.start(StandardContext.java:4704)

   at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:799)

   at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:779)

   at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:601)

   at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:675)

   at org.apache.catalina.startup.HostConfig.deployDescriptors(HostConfig.java:601)

   at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:502)

   at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1315)

   at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:324)

   at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)

   at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1061)

   at org.apache.catalina.core.StandardHost.start(StandardHost.java:840)

   at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)

   at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)

   at org.apache.catalina.core.StandardService.start(StandardService.java:525)

   at org.apache.catalina.core.StandardServer.start(StandardServer.java:754)

   at org.apache.catalina.startup.Catalina.start(Catalina.java:595)

   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

   at java.lang.reflect.Method.invoke(Method.java:597)

   at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)

   at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)

Caused by: java.lang.ClassNotFoundException: org.apache.commons.configuration.Configuration

   at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1680)

   at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1526)

   ... 36 more






/var/log/oozie/catalina.out에 있는 에러 해결

REASON: java.lang.NoClassDefFoundError: org/apache/commons/configuration/Configuration



폴더 소유자와 권한을 변경
chown -R hadoop:hadoop /usr/lib/oozie
chmod -R 755 /usr/lib/oozie




commons-configuration-1.6.jar파일을 복사 
/usr/lib/oozie/oozie-server/lib 폴더에 복사




이렇게 시도 아래와 같은 에러는 표시안됨 (해결 된것으로 보임)

해결하는데 참고한 페이지







위 문제 해결 후 다시 에러 발생 해결


ooziedb를 실행해야 할 거 같아 실행 했는데 아래와 같은 에러가 나와 해결

[hadoop@localhost usr]$

[hadoop@localhost usr]$ su root


[root@localhost usr]# /usr/lib/oozie/bin/ooziedb.sh create -sqlfile oozie.sql -run Validate DB Connection


Validate DB Connection

DONE

Check DB schema does not exist

DONE

Check OOZIE_SYS table does not exist

DONE

Create SQL schema

DONE

Create OOZIE_SYS table

DONE


Oozie DB has been created for Oozie version '3.2.0.23'



The SQL commands have been written to: oozie.sql

[root@localhost usr]# su hadoop






Apr 30, 2013 1:58:15 AM org.apache.catalina.core.AprLifecycleListener init

INFO: Loaded APR based Apache Tomcat Native library 1.1.27.

Apr 30, 2013 1:58:15 AM org.apache.catalina.core.AprLifecycleListener init

INFO: APR capabilities: IPv6 [true], sendfile [true], accept filters [false], random [true].

Apr 30, 2013 1:58:15 AM org.apache.catalina.core.AprLifecycleListener lifecycleEvent

INFO: Failed to initialize the SSLEngine.

Apr 30, 2013 1:58:15 AM org.apache.coyote.http11.Http11AprProtocol init

INFO: Initializing Coyote HTTP/1.1 on http-11000

Apr 30, 2013 1:58:15 AM org.apache.catalina.startup.Catalina load

INFO: Initialization processed in 929 ms

Apr 30, 2013 1:58:15 AM org.apache.catalina.core.StandardService start

INFO: Starting service Catalina

Apr 30, 2013 1:58:15 AM org.apache.catalina.core.StandardEngine start

INFO: Starting Servlet Engine: Apache Tomcat/6.0.32

Apr 30, 2013 1:58:15 AM org.apache.catalina.startup.HostConfig deployDescriptor

INFO: Deploying configuration descriptor oozie.xml


ERROR: Oozie could not be started


REASON: org.apache.oozie.service.ServiceException: E0103: Could not load service classes, Schema 'SA' does not exist {SELECT t0.bean_type, t0.conf, t0.console_url, t0.cred, t0.data, t0.error_code, t0.error_message, t0.external_child_ids, t0.external_id, t0.external_status, t0.name, t0.retries, t0.stats, t0.tracker_uri, t0.transition, t0.type, t0.user_retry_count, t0.user_retry_interval, t0.user_retry_max, t0.end_time, t0.execution_path, t0.last_check_time, t0.log_token, t0.pending, t0.pending_age, t0.signal_value, t0.sla_xml, t0.start_time, t0.status, t0.wf_id FROM WF_ACTIONS t0 WHERE t0.bean_type = ? AND t0.id = ?} [code=30000, state=42Y07]


Stacktrace:

-----------------------------------------------------------------

org.apache.oozie.service.ServiceException: E0103: Could not load service classes, Schema 'SA' does not exist {SELECT t0.bean_type, t0.conf, t0.console_url, t0.cred, t0.data, t0.error_code, t0.error_message, t0.external_child_ids, t0.external_id, t0.external_status, t0.name, t0.retries, t0.stats, t0.tracker_uri, t0.transition, t0.type, t0.user_retry_count, t0.user_retry_interval, t0.user_retry_max, t0.end_time, t0.execution_path, t0.last_check_time, t0.log_token, t0.pending, t0.pending_age, t0.signal_value, t0.sla_xml, t0.start_time, t0.status, t0.wf_id FROM WF_ACTIONS t0 WHERE t0.bean_type = ? AND t0.id = ?} [code=30000, state=42Y07]

   at org.apache.oozie.service.Services.loadServices(Services.java:281)

   at org.apache.oozie.service.Services.init(Services.java:202)

   at org.apache.oozie.servlet.ServicesLoader.contextInitialized(ServicesLoader.java:39)

   at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4205)

   at org.apache.catalina.core.StandardContext.start(StandardContext.java:4704)

   at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:799)

   at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:779)

   at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:601)

   at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:675)

   at org.apache.catalina.startup.HostConfig.deployDescriptors(HostConfig.java:601)

   at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:502)

   at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1315)

   at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:324)

   at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)

   at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1061)

   at org.apache.catalina.core.StandardHost.start(StandardHost.java:840)

   at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)

   at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)

   at org.apache.catalina.core.StandardService.start(StandardService.java:525)

   at org.apache.catalina.core.StandardServer.start(StandardServer.java:754)

   at org.apache.catalina.startup.Catalina.start(Catalina.java:595)

   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

   at java.lang.reflect.Method.invoke(Method.java:597)

   at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)

   at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)

Caused by: <openjpa-2.1.0-r422266:1071316 fatal general error> org.apache.openjpa.persistence.PersistenceException: Schema 'SA' does not exist {SELECT t0.bean_type, t0.conf, t0.console_url, t0.cred, t0.data, t0.error_code, t0.error_message, t0.external_child_ids, t0.external_id, t0.external_status, t0.name, t0.retries, t0.stats, t0.tracker_uri, t0.transition, t0.type, t0.user_retry_count, t0.user_retry_interval, t0.user_retry_max, t0.end_time, t0.execution_path, t0.last_check_time, t0.log_token, t0.pending, t0.pending_age, t0.signal_value, t0.sla_xml, t0.start_time, t0.status, t0.wf_id FROM WF_ACTIONS t0 WHERE t0.bean_type = ? AND t0.id = ?} [code=30000, state=42Y07]

FailedObject: 1 [org.apache.openjpa.util.StringId] [java.lang.String]

   at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:4869)

   at org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.java:4829)

   at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:136)

   at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:86)

   at org.apache.openjpa.jdbc.kernel.JDBCStoreManager.initialize(JDBCStoreManager.java:338)

   at org.apache.openjpa.kernel.DelegatingStoreManager.initialize(DelegatingStoreManager.java:112)

   at org.apache.openjpa.kernel.ROPStoreManager.initialize(ROPStoreManager.java:57)

   at org.apache.openjpa.kernel.BrokerImpl.initialize(BrokerImpl.java:1022)

   at org.apache.openjpa.kernel.BrokerImpl.find(BrokerImpl.java:980)

   at org.apache.openjpa.kernel.BrokerImpl.find(BrokerImpl.java:897)

   at org.apache.openjpa.kernel.DelegatingBroker.find(DelegatingBroker.java:223)

   at org.apache.openjpa.persistence.EntityManagerImpl.find(EntityManagerImpl.java:478)

   at org.apache.oozie.service.JPAService.init(JPAService.java:159)

   at org.apache.oozie.service.Services.setServiceInternal(Services.java:358)

   at org.apache.oozie.service.Services.setService(Services.java:344)

   at org.apache.oozie.service.Services.loadServices(Services.java:278)

   ... 26 more

Caused by: org.apache.openjpa.lib.jdbc.ReportingSQLException: Schema 'SA' does not exist {SELECT t0.bean_type, t0.conf, t0.console_url, t0.cred, t0.data, t0.error_code, t0.error_message, t0.external_child_ids, t0.external_id, t0.external_status, t0.name, t0.retries, t0.stats, t0.tracker_uri, t0.transition, t0.type, t0.user_retry_count, t0.user_retry_interval, t0.user_retry_max, t0.end_time, t0.execution_path, t0.last_check_time, t0.log_token, t0.pending, t0.pending_age, t0.signal_value, t0.sla_xml, t0.start_time, t0.status, t0.wf_id FROM WF_ACTIONS t0 WHERE t0.bean_type = ? AND t0.id = ?} [code=30000, state=42Y07]

   at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(LoggingConnectionDecorator.java:281)

   at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(LoggingConnectionDecorator.java:261)

   at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.access$000(LoggingConnectionDecorator.java:72)

   at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator$LoggingConnection.prepareStatement(LoggingConnectionDecorator.java:313)

   at org.apache.openjpa.lib.jdbc.DelegatingConnection.prepareStatement(DelegatingConnection.java:155)

   at org.apache.openjpa.lib.jdbc.ConfiguringConnectionDecorator$ConfiguringConnection.prepareStatement(ConfiguringConnectionDecorator.java:158)

   at org.apache.openjpa.lib.jdbc.DelegatingConnection.prepareStatement(DelegatingConnection.java:155)

   at org.apache.openjpa.jdbc.kernel.JDBCStoreManager$RefCountConnection.prepareStatement(JDBCStoreManager.java:1655)

   at org.apache.openjpa.lib.jdbc.DelegatingConnection.prepareStatement(DelegatingConnection.java:144)

   at org.apache.openjpa.jdbc.sql.SQLBuffer.prepareStatement(SQLBuffer.java:561)

   at org.apache.openjpa.jdbc.sql.SQLBuffer.prepareStatement(SQLBuffer.java:541)

   at org.apache.openjpa.jdbc.sql.SelectImpl.prepareStatement(SelectImpl.java:479)

   at org.apache.openjpa.jdbc.sql.SelectImpl.execute(SelectImpl.java:420)

   at org.apache.openjpa.jdbc.sql.SelectImpl.execute(SelectImpl.java:382)

   at org.apache.openjpa.jdbc.kernel.JDBCStoreManager.getInitializeStateResult(JDBCStoreManager.java:576)

   at org.apache.openjpa.jdbc.kernel.JDBCStoreManager.initializeState(JDBCStoreManager.java:378)

   at org.apache.openjpa.jdbc.kernel.JDBCStoreManager.initialize(JDBCStoreManager.java:333)

   ... 37 more

-----------------------------------------------------------------










퍼미션 에러가 난경우
계정을 root로 변경후 실행 했음

[hadoop@localhost usr]$ /usr/lib/oozie/bin/ooziedb.sh create -sqlfile oozie.sql -run Validate DB Connection


Validate DB Connection

2013-04-30 06:34:43.498 GMT Thread[main,5,main] java.io.FileNotFoundException: derby.log (Permission denied)

----------------------------------------------------------------

2013-04-30 06:34:43.685 GMT:

Booting Derby version The Apache Software Foundation - Apache Derby - 10.6.1.0 - (938214): instance a816c00e-013e-59a6-7dab-0000026d4bc8

on database directory /home/hadoop/oozie/oozie-db   with class loader sun.misc.Launcher$AppClassLoader@35a16869


Database Class Loader started - derby.database.classpath=''

DONE

Check DB schema does not exist

DONE

Check OOZIE_SYS table does not exist

DONE

Create SQL schema


Error: java.io.FileNotFoundException: oozie.sql (Permission denied)


--------------------------------------

org.apache.commons.lang.exception.NestableRuntimeException: java.io.FileNotFoundException: oozie.sql (Permission denied)

   at org.apache.openjpa.lib.util.Files.getWriter(Files.java:241)

   at org.apache.openjpa.jdbc.meta.MappingTool.run(MappingTool.java:997)

   at org.apache.openjpa.jdbc.meta.MappingTool$1.run(MappingTool.java:936)

   at org.apache.openjpa.lib.conf.Configurations.launchRunnable(Configurations.java:741)

   at org.apache.openjpa.lib.conf.Configurations.runAgainstAllAnchors(Configurations.java:731)

   at org.apache.openjpa.jdbc.meta.MappingTool.main(MappingTool.java:931)

   at org.apache.oozie.tools.OozieDBCLI.createUpgradeDB(OozieDBCLI.java:569)

   at org.apache.oozie.tools.OozieDBCLI.createDB(OozieDBCLI.java:170)

   at org.apache.oozie.tools.OozieDBCLI.run(OozieDBCLI.java:112)

   at org.apache.oozie.tools.OozieDBCLI.main(OozieDBCLI.java:63)

Caused by: java.io.FileNotFoundException: oozie.sql (Permission denied)

   at java.io.FileOutputStream.open(Native Method)

   at java.io.FileOutputStream.<init>(FileOutputStream.java:194)

   at java.io.FileOutputStream.<init>(FileOutputStream.java:145)

   at java.io.FileWriter.<init>(FileWriter.java:73)

   at org.apache.openjpa.lib.util.Files.getWriter(Files.java:239)

   ... 9 more

--------------------------------------


[hadoop@localhost usr]$










su hadoop
/usr/lib/oozie/bin/oozie-start.sh

웹페이지 열린 스샷






--


oozie admin -oozie http://$oozie.full.hostname:11000/oozie -status







제대로 실행 되었을 때 로그 기록


catalina.out

Apr 30, 2013 2:39:59 AM org.apache.catalina.core.AprLifecycleListener init

INFO: Loaded APR based Apache Tomcat Native library 1.1.27.

Apr 30, 2013 2:39:59 AM org.apache.catalina.core.AprLifecycleListener init

INFO: APR capabilities: IPv6 [true], sendfile [true], accept filters [false], random [true].

Apr 30, 2013 2:39:59 AM org.apache.catalina.core.AprLifecycleListener lifecycleEvent

INFO: Failed to initialize the SSLEngine.

Apr 30, 2013 2:40:00 AM org.apache.coyote.http11.Http11AprProtocol init

INFO: Initializing Coyote HTTP/1.1 on http-11000

Apr 30, 2013 2:40:00 AM org.apache.catalina.startup.Catalina load

INFO: Initialization processed in 934 ms

Apr 30, 2013 2:40:00 AM org.apache.catalina.core.StandardService start

INFO: Starting service Catalina

Apr 30, 2013 2:40:00 AM org.apache.catalina.core.StandardEngine start

INFO: Starting Servlet Engine: Apache Tomcat/6.0.32

Apr 30, 2013 2:40:00 AM org.apache.catalina.startup.HostConfig deployDescriptor

INFO: Deploying configuration descriptor oozie.xml

Apr 30, 2013 2:40:08 AM org.apache.catalina.startup.HostConfig deployDirectory

INFO: Deploying web application directory ROOT

Apr 30, 2013 2:40:08 AM org.apache.coyote.http11.Http11AprProtocol start

INFO: Starting Coyote HTTP/1.1 on http-11000

Apr 30, 2013 2:40:08 AM org.apache.catalina.startup.Catalina start

INFO: Server startup in 8261 ms






derby.log

----------------------------------------------------------------

2013-04-30 06:40:05.183 GMT:

Booting Derby version The Apache Software Foundation - Apache Derby - 10.6.1.0 - (938214): instance a816c00e-013e-59ab-6580-0000021ad148

on database directory /home/hadoop/oozie/oozie-db   with class loader WebappClassLoader

 context: /oozie

 delegate: false

 repositories:

/WEB-INF/classes/

----------> Parent Classloader:

org.apache.catalina.loader.StandardClassLoader@48d19bc8



Database Class Loader started - derby.database.classpath=''






oozie.log 이 파일을 주기적으로 계속 파일이 쓰이며 oozie가 정상적으로 실행시 표시되는 로그

2013-04-30 02:40:01,395 DEBUG CallableQueueService:545 - USER[-] GROUP[-] Adding interrupt type [kill]

2013-04-30 02:40:01,411 DEBUG CallableQueueService:545 - USER[-] GROUP[-] Adding interrupt type [resume]

2013-04-30 02:40:01,411 DEBUG CallableQueueService:545 - USER[-] GROUP[-] Adding interrupt type [suspend]

2013-04-30 02:40:01,411 DEBUG CallableQueueService:545 - USER[-] GROUP[-] Adding interrupt type [bundle_kill]

2013-04-30 02:40:01,411 DEBUG CallableQueueService:545 - USER[-] GROUP[-] Adding interrupt type [bundle_resume]

2013-04-30 02:40:01,411 DEBUG CallableQueueService:545 - USER[-] GROUP[-] Adding interrupt type [bundle_suspend]

2013-04-30 02:40:01,412 DEBUG CallableQueueService:545 - USER[-] GROUP[-] Adding interrupt type [coord_kill]

2013-04-30 02:40:01,413 DEBUG CallableQueueService:545 - USER[-] GROUP[-] Adding interrupt type [coord_change]

2013-04-30 02:40:01,413 DEBUG CallableQueueService:545 - USER[-] GROUP[-] Adding interrupt type [coord_resume]

2013-04-30 02:40:01,413 DEBUG CallableQueueService:545 - USER[-] GROUP[-] Adding interrupt type [coord_suspend]

2013-04-30 02:40:01,530  WARN ConfigUtils:542 - USER[-] GROUP[-] Using a deprecated configuration property [oozie.service.AuthorizationService.security.enabled], should use [oozie.service.AuthorizationService.authorization.enabled].  Please delete the deprecated property in order for the new property to take effect.

2013-04-30 02:40:01,534  INFO AuthorizationService:539 - USER[-] GROUP[-] Oozie running with security enabled

2013-04-30 02:40:01,536  INFO HadoopAccessorService:539 - USER[-] GROUP[-] JOB_TRACKER_WHITELIST :[ ], Total entries :0

2013-04-30 02:40:01,540  INFO HadoopAccessorService:539 - USER[-] GROUP[-] NAME_NODE_WHITELIST :[ ], Total entries :0

2013-04-30 02:40:01,540  INFO HadoopAccessorService:539 - USER[-] GROUP[-] Oozie Kerberos Authentication [disabled]

2013-04-30 02:40:07,785  INFO JPAService:539 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] All entities initialized

2013-04-30 02:40:07,787  INFO JPAService:539 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] JPA configuration: DriverClassName=org.apache.derby.jdbc.EmbeddedDriver,Url=jdbc:derby:/home/hadoop/oozie/oozie-db;create=true,Username=hadoop,Password=hadoop,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false

2013-04-30 02:40:07,950  INFO Services:539 - Initialized

2013-04-30 02:40:07,953  INFO Services:539 - Running with JARs for Hadoop version [1.1.2.23]

2013-04-30 02:40:07,953  INFO Services:539 - Oozie System ID [oozie-hado] started!






oozie-jpa.log 계속 로그가 쌓이는 파일이고 정상 실행시 출력된 로그 일부

2013-04-30 02:31:55,424  INFO Tool:81 - Recording mapping and schema changes.

2013-04-30 02:32:46,129  INFO JDBC:81 - Using dictionary class "org.apache.openjpa.jdbc.sql.DerbyDictionary".

2013-04-30 02:32:46,325  INFO Tool:81 - Mapping tool running on type "class org.apache.oozie.client.rest.JsonSLAEvent" with action "buildSchema".

2013-04-30 02:32:46,772  INFO Tool:81 - Mapping tool running on type "class org.apache.oozie.client.rest.JsonWorkflowJob" with action "buildSchema".

2013-04-30 02:32:46,787  INFO Tool:81 - Mapping tool running on type "class org.apache.oozie.WorkflowJobBean" with action "buildSchema".

2013-04-30 02:32:46,805  INFO Tool:81 - Mapping tool running on type "class org.apache.oozie.client.rest.JsonCoordinatorAction" with action "buildSchema".

2013-04-30 02:32:46,817  INFO Tool:81 - Mapping tool running on type "class org.apache.oozie.BundleJobBean" with action "buildSchema".

2013-04-30 02:32:46,858  INFO Tool:81 - Mapping tool running on type "class org.apache.oozie.BundleActionBean" with action "buildSchema".

2013-04-30 02:32:46,889  INFO Tool:81 - Mapping tool running on type "class org.apache.oozie.client.rest.JsonCoordinatorJob" with action "buildSchema".

2013-04-30 02:32:46,913  INFO Tool:81 - Mapping tool running on type "class org.apache.oozie.client.rest.JsonBundleJob" with action "buildSchema".

2013-04-30 02:32:46,913  INFO Tool:81 - Mapping tool running on type "class org.apache.oozie.CoordinatorJobBean" with action "buildSchema".

2013-04-30 02:32:46,936  INFO Tool:81 - Mapping tool running on type "class org.apache.oozie.WorkflowActionBean" with action "buildSchema".

2013-04-30 02:32:46,964  INFO Tool:81 - Mapping tool running on type "class org.apache.oozie.SLAEventBean" with action "buildSchema".

2013-04-30 02:32:46,981  INFO Tool:81 - Mapping tool running on type "class org.apache.oozie.client.rest.JsonWorkflowAction" with action "buildSchema".

2013-04-30 02:32:46,981  INFO Tool:81 - Mapping tool running on type "class org.apache.oozie.CoordinatorActionBean" with action "buildSchema".

2013-04-30 02:32:47,008  INFO Tool:81 - Recording mapping and schema changes.

2013-04-30 02:32:47,726  INFO JDBC:81 - Using dictionary class "org.apache.openjpa.jdbc.sql.DerbyDictionary".

2013-04-30 02:32:47,735  INFO Tool:81 - Mapping tool running on type "class org.apache.oozie.client.rest.JsonSLAEvent" with action "buildSchema".

2013-04-30 02:32:47,841  INFO Tool:81 - Mapping tool running on type "class org.apache.oozie.client.rest.JsonWorkflowJob" with action "buildSchema".

2013-04-30 02:32:47,866  INFO Tool:81 - Mapping tool running on type "class org.apache.oozie.WorkflowJobBean" with action "buildSchema".

2013-04-30 02:32:47,895  INFO Tool:81 - Mapping tool running on type "class org.apache.oozie.client.rest.JsonCoordinatorAction" with action "buildSchema".

2013-04-30 02:32:47,926  INFO Tool:81 - Mapping tool running on type "class org.apache.oozie.BundleJobBean" with action "buildSchema".

2013-04-30 02:32:47,991  INFO Tool:81 - Mapping tool running on type "class org.apache.oozie.BundleActionBean" with action "buildSchema".

2013-04-30 02:32:48,015  INFO Tool:81 - Mapping tool running on type "class org.apache.oozie.client.rest.JsonCoordinatorJob" with action "buildSchema".

2013-04-30 02:32:48,044  INFO Tool:81 - Mapping tool running on type "class org.apache.oozie.client.rest.JsonBundleJob" with action "buildSchema".

2013-04-30 02:32:48,048  INFO Tool:81 - Mapping tool running on type "class org.apache.oozie.CoordinatorJobBean" with action "buildSchema".

2013-04-30 02:32:48,073  INFO Tool:81 - Mapping tool running on type "class org.apache.oozie.WorkflowActionBean" with action "buildSchema".

2013-04-30 02:32:48,111  INFO Tool:81 - Mapping tool running on type "class org.apache.oozie.SLAEventBean" with action "buildSchema".

2013-04-30 02:32:48,117  INFO Tool:81 - Mapping tool running on type "class org.apache.oozie.client.rest.JsonWorkflowAction" with action "buildSchema".

2013-04-30 02:32:48,117  INFO Tool:81 - Mapping tool running on type "class org.apache.oozie.CoordinatorActionBean" with action "buildSchema".

2013-04-30 02:32:48,153  INFO Tool:81 - Recording mapping and schema changes.

2013-04-30 02:40:03,770  INFO Runtime:81 - Starting OpenJPA 2.1.0

2013-04-30 02:40:05,858  INFO JDBC:81 - Using dictionary class "org.apache.openjpa.jdbc.sql.DerbyDictionary" (Apache Derby 10.6.1.0 - (938214) ,Apache Derby Embedded JDBC Driver 10.6.1.0 - (938214)).

2013-04-30 02:40:06,227  WARN Enhance:89 - Creating subclass for "[class org.apache.oozie.util.db.ValidateConnectionBean]". This means that your application will be less efficient and will consume more memory than it would if you ran the OpenJPA enhancer. Additionally, lazy loading will not be available for one-to-one and many-to-one persistent attributes in types using field access; they will be loaded eagerly instead.






신고





Posted by 파란물

티스토리 툴바