Running Delivered and Custom App Engine Process using PSAE command line crashes when terminating.

Application Engine program XXX ended normally

15339: 1404097409: PSPAL::Abort: Unexpected signal received
15339: 1404097409: PSPAL::Abort: Location: /vob/peopletools/src/pspal/exception_sigaction.cpp:494: RecoverableSignalHandler
15339: 1404097409: PSPAL::Abort: Generating process state report to /opt/pshome/logs/psae.15339/process_state.txt
15339: 1404097409: PSPAL::Abort: Terminating process now.

On examining the process_state.txt, found the following

(Looking for PSPAL::)

#9  0x00007f18e0e52c3b in STWGCTimer::register_gc_start(long) () from /opt/tools/jre/lib/amd64/server/libjvm.so
#10 0x00007f18e117d8a8 in PSScavenge::invoke_no_policy() () from /opt/tools/jre/lib/amd64/server/libjvm.so
#11 0x00007f18e117ef21 in PSScavenge::invoke() () from /opt/tools/jre/lib/amd64/server/libjvm.so
#12 0x00007f18e1136020 in ParallelScavengeHeap::failed_mem_allocate(unsigned long) () from /opt/tools/jre/lib/amd64/server/libjvm.so

It seems that issue is happening, when the application engine program that invokes the java code and at the time of terminating trying to clean the JVM memory. When the JVM memory area points to invalid locations, it crashes the JVM. This is observed in 8.53.12 and currently there is no official solution, other than POC to fix this issue. You will need to open a support case to get the POC. In POC, libpsjni.so has been modified so that it does not try to clear the java memory when exiting the program.

SQLExec : Return: 8015 - Bind value is too long

You get this error in an online page or while running a Application engine program. This error happens when you try to insert more than 254 characters in a long field using sqlexec and do not use %TextIn meta sql.

Resolution

Use %TextIn meta-sql for the bind variable that is used for inserting into a long field. For e.g. %TextIn(:1)

%TextIn is documented in peoplebooks and is mandatory for all insertions/update of LongChar fields using sqlexec for all database platforms.

Here are some resolutions that discusses this issue in Metalink – Oracle support site.

E-AE Application Engine PeopleCode Step with SQLExec Receives Error; return code 8015 "Bind value is too long" [ID 889806.1]

E-PC:"Bind value is too long" Error When Using SQLExec to Insert into Long Char Field [ID 620874.1]

Could not sign on to database xxxx with user yyyy for app engine program

When trying to run appengine program from command line (psae), you are getting this message

“Could not sign on to database xxxx with user yyyy for app engine program”

One possible reason is that psae needs database name in upper case. If you are using lower case name then the above message appears.

Also PS_SERVER_CFG must contain the fully qualified name of a correctly configured Process Scheduler PSPRCS.CFG file

For e.g.

 

PS_SERVER_CFG=$PS_HOME/appserv/prcs/<domainname>/psprcs.cfg;export PS_SERVER_CFG

psae -CT ORACLE -CD dbname-CO userid -CP password -R runcontrolid -I 0 -AI <appengine progname>

Also When PeopleSoft Application Engine runs from the command line, it resolves %PS_SERVDIR% to the value of the environment variable PS_SERVDIR instead of the parent directory of a Process Scheduler configuration.

Configuration Settings for Tracing

Online Process

Set the following in psappsrv.cfg, or for a single user session in PIA using trace=y

“31” is the recommended value when tracing with “TraceSQL”.
“1984” is the recommended value when tracing with “TracePC”.

Batch Process

Application Engine

"135" is the recommended value when tracing with "TraceAE"

if peoplecode tracing is needed, please also set

TraceSQL= 31

TracePC = 1984

You can also do this in process definition of Application Engine to affect this for only 1 program.

Override options : Append : -TRACE 135 -TOOLSTRACESQL 31 -TOOLSTRACEPC 1984

To see which processes are set with this options, run this sql.

SELECT PRCSNAME, PARMLIST FROM PS_PRCSDEFN WHERE UPPER(PARMLIST) LIKE '%TRACE%' AND PRCSTYPE = 'Application Engine';

To generate Database Level Trace, Use TraceAE=2183. This will create file within the "UDUMP" directory on the database server.

It will contain details of each SQL statement that was executed on the database including its runtime execution plan. You can then use this as a input to tkprof for generating formatted trace report. However this will not capture bind variables. To capture bind variables, following trigger is needed.

CREATE OR REPLACE TRIGGER MYDB.SET_TRACE_POCALC

BEFORE UPDATE OF RUNSTATUS ON MYDB.PSPRCSRQST

FOR EACH ROW

WHEN ( NEW.runstatus = 7

AND OLD.runstatus != 7

AND NEW.prcstype = 'Application Engine'

AND NEW.prcsname = 'PO_PO_CALC'

)

BEGIN

EXECUTE IMMEDIATE

'ALTER SESSION SET TIMED_STATISTICS = TRUE';

EXECUTE IMMEDIATE

'ALTER SESSION SET MAX_DUMP_FILE_SIZE = UNLIMITED';

EXECUTE IMMEDIATE

'ALTER SESSION SET TRACEFILE_IDENTIFIER = ''POCALC''';

EXECUTE IMMEDIATE

'ALTER SESSION SET EVENTS = ''10046 TRACE NAME CONTEXT FOREVER, LEVEL 12''';

END;

/

Cobol Process

TraceSQL=128

SQR Process

A database level trigger is the only way for generating SQR trace.

Sample trigger script.

CREATE OR REPLACE TRIGGER MYDB.SET_TRACE_INS6000

BEFORE UPDATE OF RUNSTATUS ON MYDB.PSPRCSRQST

FOR EACH ROW

WHEN ( NEW.runstatus = 7

AND OLD.runstatus != 7

AND NEW.prcstype = 'SQR REPORT'

AND NEW.prcsname = 'INS6000'

)

BEGIN

EXECUTE IMMEDIATE

'ALTER SESSION SET TIMED_STATISTICS = TRUE';
EXECUTE IMMEDIATE

'ALTER SESSION SET MAX_DUMP_FILE_SIZE = UNLIMITED';
EXECUTE IMMEDIATE

'ALTER SESSION SET TRACEFILE_IDENTIFIER = ''INS6000''';
EXECUTE IMMEDIATE

'ALTER SESSION SET EVENTS = ''10046 TRACE NAME CONTEXT FOREVER, LEVEL 12''';

END;

/

Once the raw database trace is captured, execute the program “tkprof” with following sort options:

tkprof <trace_input_file> <rpt_output_file> sys=no explain=<user_id>/<password> sort=exeela,fchela,prscpu,execpu,fchcpu

For more information, Read peoplesoft resolution 201049233: E-ORACLE:10g Master Performance Solution for Oracle 10g and download the red paper attached to it.

Application Engine Aborts if it has state record that is not present in the Database.

If you have a application engine program that has define a state record and it is not defined in the App designer it causes core dump when trying to run the program. This can happen if you have migrated a app engine program from DEV to other environment and forgot to add the state record to project. Here is a query that can identify all such app engine programs.

 

select * from PSAEAPPLSTATE a where  not exists (select 'x' from psrecdefn b where a.AE_STATE_RECNAME = b.recname).

This was observed in PT 8.48.16 and running the App engine program using command line on HP*UX 11.11 server. Database: Oracle 9.2.0.8 64 bit on HP*UX.

Here is the actual error message.

Executing PS Application Engine XXXX program which will use run control XXXX
aCC runtime: pure virtual function called for class "IPSRecBuf".
24493: 1214837730: PSPAL::Abort: Unrecoverable signal received
24493: 1214837730: PSPAL::Abort: Location: /vob/peopletools/src/pspal/exception_sigaction.cpp:553: UnrecoverableSignalHandler
24493: 1214837730: PSPAL::Abort: Generating process state report to /xxxx/xxxx/xxxx/LOGS/psae.24493/process_state.txt
24493: 1214837730: PSPAL::Abort: Terminating process now.
./xxxx.sh[67]: 24493 Abort(coredump)

 

Keyword: IPSRecBuf