We had SMON recovery crash with ORA-00607&ORA-00600: internal error code, arguments: [kdBlkCheckError]on 11.2.0.3.0 db on linux system.

Here is the all related error messages and solution:

Step 1: Check alert log

–Error Message
Block recovery from logseq 46434, block 825 to scn 14782663039152
Recovery of Online Redo Log: Thread 1 Group 50 Seq 46434 Reading mem 0
Mem# 0: +DATA/x<db_sid>/onlinelog/group_50.1918.898308101
Block recovery completed at rba 46434.847.16, scn 3441.3680573617
ORACLE Instance <DB_SID> (pid = 32) – Error 607 encountered while recovering transaction (447, 31) on object 197036643.
Errors in file /u01/app/oracle/diag/rdbms/<db_sid>/<DB_SID>/trace/<DB_SID>_smon_11855.trc:
ORA-00607: Internal error occurred while making a change to a data block
ORA-00600: internal error code, arguments: [kdBlkCheckError], [1092], [339476], [6101], [], [], [], [], [], [], [], []
Dumping diagnostic data in directory=[cdmp_20160728141255], requested by (instance=1, osid=11855 (SMON)), summary=[incident=436122].
Errors in file /u01/app/oracle/diag/rdbms/<db_sid>/<DB_SID>/trace/<DB_SID>_smon_11855.trc (incident=436123):
ORA-00600: internal error code, arguments: [kdBlkCheckError], [1092], [339476], [6101], [], [], [], [], [], [], [], []
Incident details in: /u01/app/oracle/diag/rdbms/<db_sid>/<DB_SID>/incident/incdir_436123/<DB_SID>_smon_11855_i436123.trc
Thu Agu 01 14:12:57 2016
PMON (ospid: 11772): terminating the instance due to error 474
Step 2: See what happens on your system

–login as oracle user and run adrci utulity
$ adrci

ADRCI: Release 11.2.0.3.0 – Production on Fri Aug 01 08:56:27 2016

Copyright (c) 1982, 2011, Oracle and/or its affiliates. All rights reserved.

ADR base = “/u01/app/oracle”

–listing home
adrci> show home
ADR Homes:
diag/asmtool/user_oracle/host_2223417381_80
diag/rdbms/<db_sid>/<DB_SID>
diag/clients/user_oracle/host_2223417381_80
diag/clients/user_oracle/host_2223417381_11
diag/tnslsnr/xx/listener_1521
diag/tnslsnr/xx/listener_1522
diag/diagtool/user_oracle/host_2223417381_80
diag/asm/user_oracle/host_2223417381_80

–setting home
adrci> set home diag/rdbms/<db_sid>/<DB_SID>

–check problem
adrci> show problem

ADR Home = /u01/app/oracle/diag/rdbms/<db_sid>/<DB_SID>:
*************************************************************************
PROBLEM_ID PROBLEM_KEY LAST_INCIDENT LASTINC_TIME
——————– ———————————————————– ——————– —————————————-
1 ORA 600 [kdBlkCheckError] 510335 2016-01-08 17:19:43.204000 +03:00
1 rows fetched
–check errors.In this case:
– problem id –> 1
– error is –> ORA 600 [kdBlkCheckError]
– last_incident –> 510335
– lastinc_time –> 2016-01-08 17:19:43.204000 +03:00

–get details about error
adrci> show incident -p “problem_key=’ORA 600 [kdBlkCheckError]'”

ADR Home = /u01/app/oracle/diag/rdbms/<db_sid>/<DB_SID>:
*************************************************************************
INCIDENT_ID PROBLEM_KEY CREATE_TIME
——————– ———————————————————– —————————————-
510335 ORA 600 [kdBlkCheckError] 2016-01-08 17:19:43.204000 +03:00
1 rows fetched

— get details about what happen on this error
adrci>show incident -mode detail -p “incident_id=411533”
ADR Home = /u01/app/oracle/diag/rdbms/<db_sid>/<DB_SID>:
*************************************************************************

**********************************************************
INCIDENT INFO RECORD 1
**********************************************************
INCIDENT_ID 411533
STATUS ready
CREATE_TIME 2016-08-01 13:39:07.944000 +03:00
PROBLEM_ID 27
CLOSE_TIME <NULL>
FLOOD_CONTROLLED none
ERROR_FACILITY ORA
ERROR_NUMBER 600
ERROR_ARG1 kdBlkCheckError
ERROR_ARG2 1092
ERROR_ARG3 339476
ERROR_ARG4 6101
ERROR_ARG5 <NULL>
ERROR_ARG6 <NULL>
ERROR_ARG7 <NULL>
ERROR_ARG8 <NULL>
ERROR_ARG9 <NULL>
ERROR_ARG10 <NULL>
ERROR_ARG11 <NULL>
ERROR_ARG12 <NULL>
SIGNALLING_COMPONENT <NULL>
SIGNALLING_SUBCOMPONENT <NULL>
SUSPECT_COMPONENT <NULL>
SUSPECT_SUBCOMPONENT <NULL>
ECID <NULL>
IMPACTS 0
PROBLEM_KEY ORA 600 [kdBlkCheckError]
FIRST_INCIDENT 411533
FIRSTINC_TIME 2016-08-01 13:39:07.944000 +03:00
LAST_INCIDENT 510335
LASTINC_TIME 2016-08-01 17:19:43.204000 +03:00
IMPACT1 0
IMPACT2 0
IMPACT3 0
IMPACT4 0
KEY_NAME SID
KEY_VALUE 2017.1
KEY_NAME ProcId
KEY_VALUE 32.1
KEY_NAME Client ProcId
KEY_VALUE oracle11@xx.local 17997_140071949428480
OWNER_ID 1
INCIDENT_FILE /u01/app/oracle/diag/rdbms/<db_sid>/<DB_SID>/incident/incdir_411533/<DB_SID>_smon_17997_i411533.trc
OWNER_ID 1
INCIDENT_FILE /u01/app/oracle/diag/rdbms/<db_sid>/<DB_SID>/trace/<DB_SID>_smon_17997.trc
1 rows fetched

Step 3: Solve the issue

–in pfile add below lines into pfile
*.event=”10513 trace name context forever, level 2″ ## disable SMON recovery
*.event=”10231 trace name context forever, level 10″ ## skip corrupt blocks during select

–start your database from pfile

–After db up&running find corrupted blocks
select * from v$database_block_corruption;

–Also use this sql
COLUMN owner FORMAT A20
COLUMN segment_name FORMAT A30

SELECT DISTINCT owner, segment_name
FROM v$database_block_corruption dbc
JOIN dba_extents e ON dbc.file# = e.file_id AND dbc.block# BETWEEN e.block_id and e.block_id+e.blocks-1
ORDER BY 1,2;

Select * from dba_object where object_id=’xxx’;

–drop or truncate related¬†¬†objects
drop index xxx

purge dba_recyclebin;
shu immediate
startup from spfile

–check database files via RMAN
RMAN> BACKUP VALIDATE CHECK LOGICAL DATABASE ARCHIVELOG ALL;

–Also use this sql
SQL->select * from v$database_block_corruption;

Advertisements