Logs
====

[ROOKAPPS@erpuatapp02 data]$ admanagedsrvctl.sh stop oacore_server3

You are running admanagedsrvctl.sh version 120.14.12020000.11

Enter the WebLogic Admin password:
Stopping oacore_server3...
Refer /u02/UATINST/fs2/inst/apps/UATDB_erpuatapp02/logs/appl/admin/log/adoacorectl.txt for details

Server specific logs are located at /u01/ROOKAPPS/fs2/FMW_Home/user_projects/domains/EBS_domain_UATDB/servers/oacore_server3/logs

admanagedsrvctl.sh: exiting with status 0

admanagedsrvctl.sh: check the logfile /u02/UATINST/fs2/inst/apps/UATDB_erpuatapp02/logs/appl/admin/log/adoacorectl.txt for more information ...

[ROOKAPPS@erpuatapp02 data]$


Error from adoacorectl.txt
==========================

================================================================================


03/21/18-18:15:09 :: admanagedsrvctl.sh version 120.14.12020000.11


Validated the passed arguments for the option ebs-get-serverstatus
oacore_server3 is in FAILED_NOT_RESTARTABLE mode.

Validated the passed arguments for the option ebs-nmstop-managedsrv

Shutting down managed server oacore_server3

Connecting to t3://erpuatapp02.ttd.com:7203 with userid weblogic ...

Cannot connect to t3://erpuatapp02.ttd.com:7203
ERROR: Unable to shutdown the managed server oacore_server3

Server specific logs are located at /u01/ROOKAPPS/fs2/FMW_Home/user_projects/domains/EBS_domain_UATDB/servers/oacore_server3/logs

03/21/18-18:15:20 :: admanagedsrvctl.sh: exiting with status 0

================================================================================


In Above logs,it is saying 

Cannot connect to t3://erpuatapp02.ttd.com:7203

Moved the ldap folder and then started the oacore_server3
=========================================================



[ROOKAPPS@erpuatapp02 data]$ admanagedsrvctl.sh start oacore_server3

You are running admanagedsrvctl.sh version 120.14.12020000.11

Enter the WebLogic Admin password:

Calling txkChkEBSDependecies.pl to perform dependency checks for oacore_server3
*** ALL THE FOLLOWING FILES ARE REQUIRED FOR RESOLVING RUNTIME ERRORS
*** Log File = /u02/UATINST/fs2/inst/apps/UATDB_erpuatapp02/logs/appl/rgf/TXK/txkChkEBSDependecies_Wed_Mar_21_18_19_29_2018/txkChkEBSDependecies_Wed_Mar_21_18_19_29_2018.log
Perl script txkChkEBSDependecies.pl got executed successfully


Starting oacore_server3...

Server specific logs are located at /u01/ROOKAPPS/fs2/FMW_Home/user_projects/domains/EBS_domain_UATDB/servers/oacore_server3/logs

admanagedsrvctl.sh: exiting with status 1

admanagedsrvctl.sh: check the logfile /u02/UATINST/fs2/inst/apps/UATDB_erpuatapp02/logs/appl/admin/log/adoacorectl.txt for more information ...

[ROOKAPPS@erpuatapp02 data]$




adoacorectl.txt
===============

03/21/18-18:19:27 :: admanagedsrvctl.sh version 120.14.12020000.11


Calling txkChkEBSDependecies.pl to perform dependency checks for oacore_server3
Perl script txkChkEBSDependecies.pl got executed successfully


Validated the passed arguments for the option ebs-get-serverstatus
oacore_server3 is in FAILED_NOT_RESTARTABLE mode.

Validated the passed arguments for the option ebs-nmstart-managedsrv
The Server oacore_server3 is not up. Proceeding to start the server.
FMW Version is 11.1.1.9
Checking if the Node Manager is already up..
Connecting to Node Manager ...
Successfully Connected to Node Manager.

The Node Manager is already up.
FMW Version is 11.1.1.9
JSSE is already enabled on oacore_server3
ERROR: Unable to start up the managed server oacore_server3

Server specific logs are located at /u01/ROOKAPPS/fs2/FMW_Home/user_projects/domains/EBS_domain_UATDB/servers/oacore_server3/logs

03/21/18-18:19:44 :: admanagedsrvctl.sh: exiting with status 1




Server logs
===========


<21 Mar, 2018 6:19:38 PM IST> <Error> <Configuration Management> <BEA-150021> <The admin server failed to authenticate the identity of the user  starting the managed server. The reason for the error is .>
<21 Mar, 2018 6:19:38 PM IST> <Emergency> <Management> <BEA-141151> <The admin server could not be reached at http://erpuatapp01.ttd.com:7003.>
<21 Mar, 2018 6:19:38 PM IST> <Info> <Configuration Management> <BEA-150018> <This server is being started in managed server independence mode in the absence of the admin server.>
<21 Mar, 2018 6:19:38 PM IST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STARTING>
<21 Mar, 2018 6:19:38 PM IST> <Info> <WorkManager> <BEA-002900> <Initializing self-tuning thread pool>
<21 Mar, 2018 6:19:38 PM IST> <Notice> <Log Management> <BEA-170019> <The server log file /u01/ROOKAPPS/fs2/FMW_Home/user_projects/domains/EBS_domain_UATDB/servers/oacore_server3/logs/oacore_server3.log is opened. All server side log events will be written to this file.>
Mar 21, 2018 6:19:42 PM oracle.security.jps.internal.idstore.util.LibOvdUtil pushLdapNamesToLibOvd
INFO: Pushed ldap name and types info to libOvd. Ldaps : DefaultAuthenticator:idstore.ldap.provideridstore.ldap.
<21 Mar, 2018 6:19:43 PM IST> <Notice> <Security> <BEA-090082> <Security initializing using security realm myrealm.>
<21 Mar, 2018 6:19:43 PM IST> <Critical> <Security> <BEA-090403> <Authentication for user  denied>
<21 Mar, 2018 6:19:43 PM IST> <Critical> <WebLogicServer> <BEA-000386> <Server subsystem failed. Reason: weblogic.security.SecurityInitializationException: Authentication for user  denied
weblogic.security.SecurityInitializationException: Authentication for user  denied
        at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.doBootAuthorization(CommonSecurityServiceManagerDelegateImpl.java:966)
        at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.initialize(CommonSecurityServiceManagerDelegateImpl.java:1054)
        at weblogic.security.service.SecurityServiceManager.initialize(SecurityServiceManager.java:888)
        at weblogic.security.SecurityService.start(SecurityService.java:141)
        at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
        Truncated. see log file for complete stacktrace
Caused By: javax.security.auth.login.FailedLoginException: [Security:090304]Authentication Failed: User  javax.security.auth.login.LoginException: [Security:090301]Password Not Supplied
        at weblogic.security.providers.authentication.LDAPAtnLoginModuleImpl.login(LDAPAtnLoginModuleImpl.java:261)
        at com.bea.common.security.internal.service.LoginModuleWrapper$1.run(LoginModuleWrapper.java:110)
        at java.security.AccessController.doPrivileged(Native Method)
        at com.bea.common.security.internal.service.LoginModuleWrapper.login(LoginModuleWrapper.java:106)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        Truncated. see log file for complete stacktrace
>
<21 Mar, 2018 6:19:43 PM IST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FAILED>
<21 Mar, 2018 6:19:43 PM IST> <Error> <WebLogicServer> <BEA-000383> <A critical service failed. The server will shut itself down>
<21 Mar, 2018 6:19:43 PM IST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FORCE_SHUTTING_DOWN>
<21 Mar, 2018 6:19:43 PM> <FINEST> <NodeManager> <Waiting for the process to die: 10994>
<21 Mar, 2018 6:19:43 PM> <INFO> <NodeManager> <Server failed during startup so will not be restarted>
<21 Mar, 2018 6:19:43 PM> <FINEST> <NodeManager> <runMonitor returned, setting finished=true and notifying waiters>



I dont understand why it says "<21 Mar, 2018 6:19:38 PM IST> <Emergency> <Management> <BEA-141151> <The admin server could not be reached at http://erpuatapp01.ttd.com:7003.>"



our admin url is erpuatapp01.ttd.com:7003

         <wls_admin_host oa_var="s_wls_admin_host">erpuatapp01</wls_admin_host>
         <wls_admin_domain oa_var="s_wls_admin_domain">ttd.com</wls_admin_domain>
         <wls_admin_user oa_var="s_wls_admin_user">weblogic</wls_admin_user>
         <wls_adminport oa_var="s_wls_adminport" oa_type="PORT" base="7001" step="1" range="-1" label="WLS Admin Server Port">7003</wls_adminport>


we are able to telnet from 2nd to admin host

[ROOKAPPS@erpuatapp02 ~]$ telnet erpuatapp01.ttd.com 7003
Trying 172.16.111.139...
Connected to erpuatapp01.ttd.com.
Escape character is '^]'.