Database Problem Or system

[latifa]

Hi Erman,

Is it an Oracle bug or system problem ? Oracle DB 9i.

10/03/2017 11:49:24 - Cause     : ORA-01115: IO error reading block from file 413 (block # 206185)
ORA-01110: data file 413: '/DATA/file3.dbf'
ORA-27091: skgfqio: unable to queue I/O
ORA-27072: skgfdisp: I/O error
SVR4 Er


regards,
Latifa

[ErmanArslansOracleBlog]

Is the OS IBM AIX?

If it is so, following is your action plan:
( we have encountered earlier in our AIX customers...)


Applies to:Oracle Database - Enterprise Edition - Version 9.2.0.8 and later
IBM AIX on POWER Systems (64-bit)
***Checked for relevance on 16-Feb-2011***

IBM AIX Based Systems (64-bit)

Symptoms

After patching the OS with Patch IBM Technology Level 6 (5300-06) or Level 7 (5300-07), the application log started reporting the following errors:

ORA-01115: IO error reading block from file <file id> (block # <block number>)
ORA-01110: data file <file id>: <file name> 
ORA-27091: skgfqio: unable to queue I/O
ORA-27072: skgfdisp: I/O error
IBM AIX RISC System/6000 Error: 5: I/O error

- The errors are not reported in the alert log file.
- Running dbv against datafiles from the error messages shows no errors.
- All the mandatory OS patches are applied.
- Oracle patch for Bug 5496862 - IO READING PROBLEMS AFTER INSTALLING IBM TECHNOLOGY LEVEL 5 (5300-05) was installed before patching the OS.

Patch IBM Technology Level 6 (5300-06) or higher was applied to the OS.

The value of maxreqs (4096) was too low. This asynchronous I/O parameter specifies the maximum number of asynchronous I/O requests that can be outstanding at any one time and has as default value 4096.

Increase maxreqs to a value greater than or equal to 8192.

Steps:
1. run aioo -a command to double check current setting for aio0 device
2. run aioo -o maxreqs=<value> to set maxreqs dynamically
3. chdev -l aio0 -a maxreqs=<value> -P to set the value of maxreqs permanently for next reboot
4. run aioo -a to confirm change
5. restart oracle

NOTE: Values that fixed the errors: 8192, 16384 or 32768.

[latifa]

it's not an AIX:

SunOS udkoar01 5.8 Generic_Virtual sun4u sparc SUNW,SPARC-Enterprise

[ErmanArslansOracleBlog]

Tell me all the details..

1)What is the db version?
2)Where is this error recorded?
3)During or after which activity you see this error recorded?
4)Send me the whole error . (it seems you cut it.. I have to see the whole SVR4 Error...)
5)What is the filesystem you are using for db files? ASM?

[latifa]

1)What is the db version?
9.2.0.6.0
2)Where is this error recorded?

during running a script to extract providers.

3)During or after which activity you see this error recorded?
after data extraction of the customer.
4)Send me the whole error . (it seems you cut it.. I have to see the whole SVR4 Error...)

10/03/2017 11:49:24 - Procedure : MAIN
10/03/2017 11:49:24 - Exception : OTHERS
10/03/2017 11:49:24 - Message   : ouverture des fichiers
10/03/2017 11:49:24 - Cause     : ORA-01115: IO error reading block from file 413 (block # 206185)
ORA-01110: data file 413: '/DATA/pod03.dbf'
ORA-27091: skgfqio: unable to queue I/O
ORA-27072: skgfdisp: I/O error
SVR4 Er

5)What is the filesystem you are using for db files? ASM?

there is no ASM.

--> That 's all the information I have.

restart File system.
restart DB
We tried to resize the datafile pod03.dbf with any success.
dbv :
DBV-00102: File I/O error on FILE (/DATA/pod03.dbf) during verification read operation (-2)

Regards,
Latifa

[ErmanArslansOracleBlog]

Okay , the version 9.2.0.6 is quite old.
We need to see the whole error. The end of this line -> "SVR4 Er..."
There should be something like: ORA-27072:skgfdisp: I/O error  SVR4 Error: 28:  blablabla...
The issue is most probably due to some OS level problem.
DBV-00102: (-2) tries to tell something to us, as well.

There are bugs recorded for this.

1)What do you have in alert log? Can you see the words after the SRVR4 Er...
2)What do you have in your OS logs? Do you have any IO errors reported at the same time?
3) What do you have in alert log and related oracle process trace files -- given in the alert log lines?
4)Try to following
  *) resize datafile and retry ->
  Resize the datafile using:
  alter database datafile datafile_name resize <1 block smaller>;
  If the resize fails with ORA-3297, it is also valid to resize the datafile to 1 block larger:
  alter database datafile datafile_name resize <1 block larger>;
  *) Un-mount the FS that contain the Datafile  --tell sysadmin to do this
      Run FSCK to fix the corrupted OS Blocks.  --tell sysadmin to do this
      Re-mount the FS   --tell sysadmin to do this
  *Ensure your fs is not %100 full

 

[latifa]

Hi Erman,

We tried the procedure you said in the end of your post. However it doesn't work (resize blocks ...smaller ... larger ... back ...). Mount FS and resize , etc ...
We decided to refresh database and reclone (autoconf tiers).
DBV actually didn't show any error.
Customer is testing

Regards,
Latifa

[ErmanArslansOracleBlog]

Okay Latifa. my suggesstions in this conditions, were just the things to try actually.. 9.2.0.6 is very old and the cause was probably a bug or a lack in DB-OS interoperability/configuration (it may be an IO limit, as well..)
Anyways, let's see the outcome...

[latifa]

Hi Erman,

Refresh of database solved the problem.

I hope that is done definitely !

Regards,
Latifa