Hotsos 2013 and some more useful OEM queries

January 31, 2013

Busy preparing for Hotsos 2013. My presentation this year is “Working with Confidence: How Sure Is the Oracle CBO about Its Cardinality Estimates, and Why Does It Matter?” I know it is a mouthful. The presentation/white paper would be up to the point with lots of pictures though :)

The only thing I have for the blog now is a revised version of a query I posted the last time. The original version was way too crude.

select   a.rollup_timestamp ,

         count(*) ,

         sum(c.cpu_count) ,

         sum(a.average*c.cpu_count)/ sum(c.cpu_count) ,

         sum(a.maximum*c.cpu_count)/ sum(c.cpu_count) ,

         sum((a.average+3*a.standard_deviation)*c.cpu_count) / sum(c.cpu_count)

from

         mgmt$metric_hourly a ,

         mgmt$target b ,

         sysman.MGMT_ECM_HW c

where    a.metric_name = 'Load'

and      a.column_label = 'CPU Utilization (%)'

and      a.target_guid = b.target_guid

and      b.target_name like 'dev%'

and      c.hostname||'.'||c.domain = b.target_name

and      c.vendor_name = 'Intel Based Hardware'

group by a.rollup_timestamp

order by 5 desc

The query now takes into account the number of CPUs on a server. I use the undocumented MGMT_ECM_HW view – it contains a wealth of information.

The query assumes that all CPUs are identical – not perfect, but much better than the original.


More Useful OEM Repository Queries

December 31, 2012

I already included a number of queries against the OEM repository in my presentation.
Now, I would like to add two more.

The first query shows an approximation of the average and max CPU utilization for a group of servers (names starting with “dev”).
Since the query assumed that each server has the same “weight”, the results are not completely correct for most environments.
They are a good start for a consolidation and/or virtualization projects though.

select
      a.rollup_timestamp           time, 
      count(*)                     number_of_servers,
      sum(a.average)/count(*)      average_load ,
      sum(a.maximum)/count(*)      max_recorded,
      sum(a.average+3*a.standard_deviation)/count(*) max_derived
from
      mgmt$metric_hourly a ,
      mgmt$target b
where
      a.metric_name = 'Load'
  and a.column_label = 'CPU Utilization (%)'
  and a.target_guid = b.target_guid
  and b.target_name like 'dev%'
group by a.rollup_timestamp
order by max_derived desc

1/31/2013 – The query above is quite crude - I just posted a better query  – http://wp.me/p1DHW2-5J

The second query shows all databases that belong to the TEST OEM group and are in ARCHIVELOG mode. The query can be modified to look at all databases or to any subset.

select *
from 
sysman.MGMT$GROUP_DERIVED_MEMBERSHIPS O ,
    sysman.MGMT$TARGET T ,
    sysman.MGMT$AVAILABILITY_CURRENT st
where    
     o.composite_target_name = 'TEST'
and  o.member_target_type IN ('oracle_database', 'rac_database')
and  ( t.target_type ='rac_database'
       or (t.target_type ='oracle_database'
           and t.type_qualifier3 != 'RACINST'))
and  o. member_target_guid = t.target_guid
and  t.target_guid = st.target_guid
and  st.availability_status = 'Target Up'
and  (not exists 
      (select
             *
       from
            sysman.mgmt$metric_current i
       where 
             i.target_guid = o.member_target_guid 
       and   metric_name   = 'archFull'
       and   metric_Column = 'archTotal'
       and   metric_label  = 'Archive Area'
       and   column_label  = 'Total Archive Area (KB)'
       and   key_value     = 'NOARCHIVELOG'
    ));

This query is not strictly based on the documentation, but it work in OEM 12c.


How to Use the New OEM 12c Metric Extensions to Enforce Enterprise-Wide Custom Policies

August 8, 2012

Oracle Enterprise Manager’s User-Defined Metrics (UDMs) were a very suitable vehicle for enforcing enterprise-wide policies. My NYOUG 2011/VirtaThon presentation and white paper contain a comprehensive example – monitoring forced logging status of all production databases.

Since UDMs are gone in OEM 12c, we’ll see how to use Metric Extensions, the successor of UDMs, to accomplish the same goal.

Metric Extensions (MEs) represent a major improvement over UDMs in terms of code maintenance and ability to deploy to large number of targets. MEs, however, do not allow us to customize the SQL query for a specific target. Even though there is a good reason behind the behavior, it requires us to substantially change the architecture we have previously used.

In the original presentation, I used one UDM ( ForcedLogging ) that had different definitions for different targets.

In OEM 12c, we’ll need to have different Metric Extensions for each of the definitions and target types.

For most single instance DB targets, we’ll use the standard “ForcedLogging” ME that is defined with this query:


select force_logging from v$database

For most RAC DB targets, we’ll use “ForcedLoggingRAC” ME that is defined with the same query:


select force_logging from v$database

And if we want to ignore the “force logging” status of a database, then we can use ForcedLoggingIgnore” ME defined with this query:


select 'YES' from v$database

The query/Metalized View that actually enforces the enterprise-wide force logging policy gets a little more complex:


CREATE MATERIALIZED VIEW "SYSMAN"."UDM_FORCED_LOGGING" ("CNT") ORGANIZATION HEAP PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255 NOCOMPRESS LOGGING STORAGE(INITIAL 65536 NEXT 1048576 MINEXTENTS 1 MAXEXTENTS 2147483645 PCTINCREASE 0 FREELISTS 1 FREELIST GROUPS 1 BUFFER_POOL DEFAULT FLASH_CACHE DEFAULT CELL_FLASH_CACHE DEFAULT) TABLESPACE "MGMT_TABLESPACE" BUILD IMMEDIATE USING INDEX REFRESH COMPLETE ON DEMAND START WITH sysdate+0 NEXT SYSDATE + 1/96 USING DEFAULT LOCAL ROLLBACK SEGMENT USING ENFORCED CONSTRAINTS DISABLE QUERY REWRITE
AS
SELECT COUNT( member_target_guid ) cnt
FROM SYSMAN.MGMT$GROUP_DERIVED_MEMBERSHIPS O ,
SYSMAN.MGMT$TARGET T ,
sysman.MGMT$AVAILABILITY_CURRENT st
WHERE o.composite_target_name = 'PRODUCTION'
AND o.member_target_type     IN ('oracle_database', 'rac_database')
AND ( t.target_type           ='rac_database'
OR (t.target_type             ='oracle_database'
AND t.type_qualifier3        != 'RACINST'))
AND o. member_target_guid     = t.target_guid
AND t.target_guid             = st.target_guid
AND st.availability_status    = 'Target Up'
AND (NOT EXISTS
(SELECT *
FROM sysman.mgmt$metric_current i
WHERE I.TARGET_GUID      = O.MEMBER_TARGET_GUID
AND metric_name          = 'ME$ForcedLogging'
AND column_label         = 'ForcedLogging'
AND Metric_Column        = 'ForcedLogging'
AND collection_timestamp > sysdate - 30/1440
AND value                = 'YES'
)
AND NOT EXISTS
(SELECT *
FROM sysman.mgmt$metric_current i
WHERE i.target_guid      = o.member_target_guid
AND metric_name          = 'ME$ForcedLoggingRAC'
AND column_label         = 'ForcedLogging'
AND Metric_Column        = 'ForcedLogging'
AND collection_timestamp > sysdate - 30/1440
AND value                = 'YES'
)
AND NOT EXISTS
(SELECT *
FROM sysman.mgmt$metric_current i
WHERE i.target_guid      = o.member_target_guid
AND metric_name          = 'ME$ForcedLoggingIgnore'
AND column_label         = 'ForcedLogging'
AND Metric_Column        = 'ForcedLogging'
AND collection_timestamp > sysdate - 30/1440
AND value                = 'YES'
));


Security considerations and challenges when querying OEM repository tables/views

June 22, 2012

OEM utilizes virtual private database (VPD) policies in addition to the standard database privileges and roles. That means that the DBA cannot just give privileges on OEM objects to a DB user, it has to take care of the VPD setting as well.

There are two ways to allow a DB user to access OEM table/views.

The first way is to configure the DB user as an OEM Administrator (Setup->Security->Administrators).Give the DB user/OEM admin at least EM_ALL_VIEWER role and the DB user is ready to access all OEM objects.

The second way is to create a regularly refreshed materialized view in the SYSMAN and grant that view to the DB user who needs the information.

This is the materialized view I use for the Forced Logging UDM in OEM 12c (Forced Logging method described in defail in my presentation and white paper) :

create materialized view "SYSMAN"."UDM_FORCED_LOGGING" tablespace "????????" pctfree 10 initrans 1 maxtrans 255 storage ( initial 64K buffer_pool default) logging using no index refresh complete start with to_date('06-22-2012 14:39:25','MM-dd-yyyy hh24:mi:ss') next sysdate + 1/96
as
  select
       count( member_target_guid ) cnt
  from
       sysman.mgmt$group_derived_memberships o ,
       sysman.mgmt$target t ,
       sysman.mgmt$availability_current st
  where
       o.composite_target_name = 'PRODUCTION_SYSTEMS'
  and  o.member_target_type     in ('oracle_database', 'rac_database')
  and  ( t.target_type           ='rac_database'
         or (t.target_type            ='oracle_database'
             and t.type_qualifier3        != 'RACINST'))
  and  o. member_target_guid     = t.target_guid
  and  t.target_guid             = st.target_guid
  and  st.availability_status    = 'Target Up'
  and  t.target_name not        in ('????????')
  and  not exists
      (select
  *
       from
  sysman.mgmt$metric_current i
       where
  i.target_guid        = o.member_target_guid
       and  metric_name          = 'ME$ForcedLogging'
       and  column_label         = 'ForcedLogging'
       and  metric_column        = 'ForcedLogging'
       and  collection_timestamp > sysdate - 20/1440
       and  value                = 'YES'
    )


Oracle Enterprise Manager Oracle Enterprise Manager Cloud Control 12c is here. Let’s look under the hood!

May 18, 2012

       There are many great features in the new OEMCC 12c, and probably as many great presentations or blog posts about them.

       This post is not about a particular new feature, it is about the internal OEMCC 12c repository structures and how to make use of them. Most, if not all, of OEM 10g/11g repository views are still present and used by OEMCC12c . Also, all the ideas and techniques I presented earlier (presentation, white paper) are still conceptually valid.

        The way the structures are populated, however, can be slightly different. User Defined Metrics (UDMs) functionality is totally overhauled in OEM 12c. The new Metrics Extensions provide a flexible, structured and efficient mechanism for creating custom metrics. The new functionality now populates Metric_Name column in mgmt$metric_current with the Metric Extension name; prior versions simply inserted the hardcoded “SQLUDM” value.  Metric_Column column in mgmt$metric_current exhibits the same pattern – in OEM 12c one can populate it with a custom value, while in prior versions it always contained a pre-defined strings ( ‘StrValue’).

        The good news is that all reports and queries discussed in my presentation and white paper are still valid, except for the previously mentioned UDM/Metrics Extension change.  It took my just a couple of hours to move all custom OEM repository structures, including the advanced tablespace forecasting, to the new 12c OEM.

        This is what needs to be changed in the ForcedLoggingAlert UDM/ Metrics Extension to work with Metrics Extensions:

and   not exists (

      select 
               * 
         from
               mgmt$metric_current i
         where
               i.target_guid           = o.member_target_guid
         and   metric_name             =  ‘ME$ForcedLogging’
         and   column_label            = 'ForcedLogging'
         and   Metric_Column           = ‘ForcedLogging’
         and   collection_timestamp    > sysdate - 20/1440
      and   value                   = 'YES‘
             )

Where ‘ME$ForcedLogging’  is the name of the Metrics Extension and ‘ForcedLogging’ is the specified metric column.


More details on my OEM Repository presentation – the tech journal for the NYOUG March 2011 meeting is now available.

October 25, 2011

The NYOUG tech journal the NYOUG March 2011 meeting is available at  http://www.nyoug.org/info/tech_journal/2011/NYOUG_TechJournal_June2011.zip.

Many great papers!  Mine starts at page 35. The revised copy of my presentation can be found at http://iiotzov.files.wordpress.com/2011/08/iotzov_oem_repository.pdf


A mistake in my presentation on OEM Repository (March 2011 NYOUG ).

August 26, 2011

Special thanks to Wayne Taylor for pointing out a mistake I made on page 6 in my presentation Unleash the Value of the Data in Your OEM Repository. A corrected and slightly extended version of the presentation can be found here.

 The following clause “to_char(rollup_timestamp,’DAY’) not in (‘SATURDAY’,'SUNDAY’)” does not have the desired effect.

 The correct clause would be “to_char(rollup_timestamp,’FMDAY’) not in (‘SATURDAY’,'SUNDAY’)”.

I actually use “to_char(rollup_timestamp,’D') not in (1,7)”, but decided to include something more user-friendly for the presentation without much testing…

On a separate note – the documentation for the OEM Repository table/view structures for OEM 11g are now available – http://download.oracle.com/docs/cd/E11857_01/em.111/e12135/toc.htm . It appears that all structures in OEM 10g are present in 11g as well. That means that we can move any custom views and packages built for OEM 10g without much effort.


Follow

Get every new post delivered to your Inbox.