Performance Metrics are easier to digest if visualized trough some Line Charts. OEM, eDB360, eAdam and other tools use them. If you already have a SQL Statement that provides the Performance Metrics you care about, and just need to generate a Line Chart for them, you can easily create a CSV file and open it with MS-Excel. But if you want to build an HTML Report out of your SQL, that is a bit harder, unless you use existing technologies. Tools like eDB360 and eAdam use Google Charts as a mechanism to easily generate such Charts. A peer asked me if we could have such functionality stand-alone, and that challenged me to create and share it.
This HTML Line Chart Report above was created with script line_chart.sql shown below. The actual chart, which includes Zoom functionality on HTML can be downloaded from this Dropbox location. Feel free to use this line_chart.sql script as a template to display your Performance Metrics. It can display several series into one Chart (example above shows only one), and by reviewing code below you will find out how easy it is to adjust to your own needs. Chart above was created using a simple query against the Oracle Sample Schema SH, but the actual use could be Performance Metrics or any other Application time series.
Enkitec’s Oracle AWR Data Mining Tool
EAdam is a free tool that extracts from an Oracle database a subset of data and metadata with the objective to perform some data mining using a separate staging Oracle database. The data extracted is relevant to Performance Evaluations (PE) and/or to Sizing and Provisioning (SP) projects. Most of the data eAdam extracts is licensed by Oracle under the Diagnostics Pack, and some under the Tuning Pack. Therefore, in order to use this eAdam tool, the source database must be licensed to use both Oracle Packs (Tuning and Diagnostics).
To a point, eAdam is similar to eDB30; both access the Data Dictionary in order to produce some reports. The key difference is that eDB360 generates all the reports doing some intensive processing at the source database, while eAdam simply extracts a set of flat files into a TAR file, using a very light-weight script, delaying all the intensive processing for a later time and on a separate staging system. This feature can be very attractive for busy systems where the amount of processing of any external monitoring tool needs to be minimized.
On the source system, eAdam only needs to execute a short script to extract the data and metadata of interest, producing a dense TAR file. On a staging system, eAdam does the heavy lifting, requiring the creation of a repository, the load of this repository and finally the computation of meaningful reports. The processing of the TAR file into the staging system is usually performed by the requestor, using a lower-level database, or a remote one.
EAdam has two primary uses, listed here in order of frequency of use: 1) Performance Evaluation (PE) of an Oracle database, and 2) Sizing and Provisioning (SP) project for an Oracle database. Of course the list of uses is not comprehensive; as you may appreciate from the objects extracted, at the very least Active Session History (ASH) can be used to view performance data in more than one dimension. The list of objects eAdam extracts as flat files from the source database includes the following:
EADAM works on 10gR2, 11gR2, and on higher releases of Oracle; and it can be used on Linux or UNIX Platforms. It has not been tested on Windows. An eAdam sample output is available at this Dropbox location; after downloading the sample output, look for the 0001_eadam36_N_dbname_index.html file and start browsing.
Instructions – Source Database
Download the tool, uncompress the master ZIP file, and look for file eadam-master/source_system/eadam_extract.sql. Review and execute this single and short script connecting to the source database as SYS. Locate the TAR file produced, and send it to the requestor.
Be aware that the TAR file produced by the extraction process can be large, so be sure you execute this extract script from a directory with at least 10 GB of free space. Common sizes of this TAR file range between 100 MB and 1 GB. Execution time for this extraction process may exceed 1 hour, depending on the size of the Data Dictionary.
Instructions – Staging Database
Be sure you have both the eAdam tool (eadam-master.zip) and the TAR file produced on a source system. Your staging database can be of equal, higher or lower release level than the source, but equal or higher is recommended. The Platform can be the same or different.
To install, load and report on the staging database, proceed with the following steps:
- Create on the staging system a file directory available to Oracle for read and write. Most probably you want to create this directory connecting to OS as Oracle and create a new directory like /home/oracle/eadam-master. Put in there the content of the eadam-master.zip file.
- Create the eAdam repository on the staging database. This step is needed only one time. Follow instructions from the readme.txt. Basically you need to execute eadam-master/stage_system/eadam_install.sql connected as SYS. This script asks for 4 parameters: Tablespace names for permanent and temporary schema objects, and the username and password of the new eAdam account. For the username I recommend eadam, but you can use any valid name.
- Load the data contained in the TAR file into the database. To do this you need first to copy the TAR file into the eadam-master/stage_system sub-directory and execute next the stage_system/eadam_load.sql script while on the stage_system sub-directory and connecting as SYS. This script asks for 4 parameters. Pass first the directory path of your stage_system sub-directory, for example /home/oracle/eadam-master/stage_system (this sub-directory must contain the TAR file). Pass next the username and password of your eadam account as you created them. Pass last the name of the TAR file to be loaded into the database.
- The load process performs some data transformations and it produces at the end an output similar to eDB360 but smaller in content. After you review the eAdam output, you may decide to generate new output for shorter time series, in such case use the eadam-master/stage_system/eadam_report.sql connecting as the eadam user. This reporting process asks for 3 parameters. Pass the EADAM_SEQ_ID which identifies your particular load (a list of values is displayed), then pass the range of dates using format YYYY-MM-DD/HH24:MI, for example 2014-07-27/17:33.
EADAM @ GitHub is available as free software. You can see its readme.txt, license.txt or any other piece of the tool before downloading it. Use this link eadam-master.zip to actually download eAdam as a compressed file.
Please post your feedback about this eAdam tool at this blog, or send and email directly to the tool author: Carlos Sierra.
Enkitec’s Oracle Database 360-degree View
EDB360 is a free tool that executes on an Oracle database and produces a compressed file which includes a large set of small Reports. This set of Reports provides a 360-degree view of an Oracle Database. EDB360 is mostly used for one of the following 3 reasons, listed here in order of frequency of use: 1) Keystone of an Oracle database Health-Check. 2) Kick-off for an Oracle database Performance Evaluation. 3) High-level view of System Resources demand and utilization for an Oracle database Sizing and Provisioning project.
Usually, Developers, Sys Admins and Consultants are not given open access to a database in a Production environment. This eDB360 free tool helps approved users to become familiar with an Oracle database in a non-intrusive way. Without installing anything on the database, the eDB360 tool connects to an Oracle database and produces a large set of flat files that can be reviewed offline while using an HTML browser or a Text editor.
EDB360 can be executed by someone with very limited access to an Oracle database (i.e. a Developer, Sys Admin or Consultant with just query access to the Data Dictionary views); or if executed by an authorized DBA, there is no actual need to provide any additional access to the Oracle database to the party requesting eDB360.
EDB360 works on 10gR2, 11gR2, and on higher releases of Oracle; and it can be used on Linux or UNIX Platforms. It has not been tested on Windows. An eDB360 sample output is available at this Dropbox location; after downloading the sample output, look for the 0001_edb360_dbname_index.html file and start browsing.
Download the eDB360 tool and review the readme.txt file included. Uncompress the eDB360 master ZIP file on the Database Server of interest. Navigate to the main eDB360 (master) directory and execute script edb360.sql connected as SYS or any other account with access to the Data Dictionary views (a DBA account is not required but it is preferred).
Execution time for eDB360 may exceed 1 hour, depending on the size of the Data Dictionary. And the size of the output may reach 1 GB, so be sure you execute this tool from a file system directory with at least 1 GB or free space. Common sizes of the output range between 10 and 100 MB.
EDB360 has only two execution parameters:
- Oracle Pack License: A big portion of the information presented by eDB360 comes from Oracle’s Automatic Workload Repository (AWR), and AWR is licensed by Oracle under the Diagnostics Pack. A small part of the output of eDB360 comes from the SQL Monitoring repository, which is part of the Oracle Tuning Pack. This parameter accepts one of 3 values: “T”, “D” or “N”. If you database is licensed under the Oracle Tuning Pack, enter then the value of “T”. If your database is not licensed to use the Oracle Tuning Pack but it is licensed to use the Oracle Diagnostics Pack, enter “D” then. If your site is not licensed on any of these two Oracle Packs, enter “N” then. Be aware that a value of “N” reduces substantially the content and value of the output. Thus the preferred parameter value is “T” (Oracle Tuning Pack).
- Days of History?: Assuming you pass a value of “T” or “D” on the 1st parameter, this second parameter defines how many days of History are accessed by eDB360 out of AWR. The default value is 31, but if your actual AWR setup preserves only 8 days (AWR default), then eDB360 will limit its content to the smaller of the two. For most cases, the default value of 31 is the right one to choose, specially when unsure about the actual AWR retention period.
# unzip edb360-master.zip # cd edb360-master # sqlplus / as sysdba SQL> @edb360.sql T 31
EDB360 @ GitHub is available as free software. You can see its readme.txt, license.txt or any other piece of the tool before downloading it. Use this link edb360-master.zip to actually download eDB360 as a compressed file.
Please post your feedback about this eDB360 tool at this blog, or send and email directly to the tool author: Carlos Sierra.
A good friend of mine recently asked me if edb360 included a section showing indexes that would benefit of a rebuild. I replied “not yet” and basically committed to add something. This topic of the needs versus the implications of rebuilding an index has been recently discussed in Richard Foote’s Blog. In my opinion, if you want to know more about indexes, Richard’s blog is one of the first stops. To my surprise and delight, I learned that we can actually use a little trick of the EXPLAIN PLAN FOR command and actually use the CBO to estimate what would be the size of an index if we were to create (or rebuild) it. In this latter blog posting Richard explains and demonstrates how it can be done.
My blog posting today is about encapsulating this cool method to quickly and cheaply estimate the size of an index if it were to be rebuilt, and put it on a free script for the Oracle community to use. So, feel free to use script below, but I need to remind you that before you jump to conclusions that this or that index should be rebuilt, read first about the actual need of doing so. You may want to include in your reading Richard Foote’s numerous postings on this topic, and also a couple of Oracle MOS notes: 989093.1 and 989186.1
---------------------------------------------------------------------------------------- -- -- File name: estimate_index_size.sql -- -- Purpose: Reports Indexes with an Actual size > Estimated size for over 1 MB -- -- Author: Carlos Sierra -- -- Version: 2014/07/18 -- -- Description: Script to very quickly and cheaply estimate the size of an index if it -- were to be rebuilt. It uses EXPLAIN PLAN FOR CREATE INDEX technique. -- It can be used on a single index, or all the indexes on a table, or -- a particular application schema, or all application schemas. It does not -- lock indexes and only updates the plan_table, which is usually a global -- temporary table. -- -- Usage: Connect to SQL*Plus as SYS or DBA account and execute without parameters. -- It will ask for optional schema owner, table name and index name. If all -- 3 are given null values then it acts on all application schemas. It -- generates a simple text report with the indexes having an estimated size -- of at least 1 MB over their actual size. -- -- Example: @estimate_index_size.sql -- -- Notes: Developed and tested on 18.104.22.168. -- -- Inspired on blog posts from Richard Foote and Connor MacDonald: -- http://richardfoote.wordpress.com/2014/04/24/estimate-index-size-with-explain-plan-i-cant-explain/#comment-116966 -- http://connormcdonald.wordpress.com/2012/05/30/index-size/ -- -- If considering index rebuilds based on the output of this script, read -- first Richard Foote's numerous blog postings about this topic. Bottom -- line: there are only a few cases where you actually need to manually -- rebuild an index. -- -- This method to estimated size of an index is far from perfect, please -- scrutinize this script before using it. You may also want to read -- Oracle MOS notes: 989093.1 and 989186.1 on this topic. -- --------------------------------------------------------------------------------------- -- SPO estimate_index_size.txt; UNDEF owner table_name index_name exclusion_list exclusion_list2; DEF exclusion_list = "('ANONYMOUS','APEX_030200','APEX_040000','APEX_SSO','APPQOSSYS','CTXSYS','DBSNMP','DIP','EXFSYS','FLOWS_FILES','MDSYS','OLAPSYS','ORACLE_OCM','ORDDATA','ORDPLUGINS','ORDSYS','OUTLN','OWBSYS')"; DEF exclusion_list2 = "('SI_INFORMTN_SCHEMA','SQLTXADMIN','SQLTXPLAIN','SYS','SYSMAN','SYSTEM','TRCANLZR','WMSYS','XDB','XS$NULL')"; VAR random1 VARCHAR2(30); VAR random2 VARCHAR2(30); EXEC :random1 := DBMS_RANDOM.string('A', 30); EXEC :random2 := DBMS_RANDOM.string('X', 30); DELETE plan_table WHERE statement_id IN (:random1, :random2); SET SERVEROUT ON; DECLARE sql_text CLOB; BEGIN FOR i IN (SELECT idx.owner, idx.index_name FROM dba_indexes idx, dba_tables tbl WHERE idx.owner = NVL(UPPER(TRIM('&&owner.')), idx.owner) -- optional schema owner name AND idx.table_name = NVL(UPPER(TRIM('&&table_name.')), idx.table_name) -- optional table name AND idx.index_name = NVL(UPPER(TRIM('&&index_name.')), idx.index_name) -- optional index name AND idx.owner NOT IN &&exclusion_list. -- exclude non-application schemas AND idx.owner NOT IN &&exclusion_list2. -- exclude more non-application schemas AND idx.index_type IN ('NORMAL', 'FUNCTION-BASED NORMAL', 'BITMAP', 'NORMAL/REV') -- exclude domain and lob AND idx.status != 'UNUSABLE' -- only valid indexes AND idx.temporary = 'N' AND tbl.owner = idx.table_owner AND tbl.table_name = idx.table_name AND tbl.last_analyzed IS NOT NULL -- only tables with statistics AND tbl.num_rows > 0 -- only tables with rows as per statistics AND tbl.blocks > 128 -- skip small tables AND tbl.temporary = 'N') LOOP BEGIN sql_text := 'EXPLAIN PLAN SET STATEMENT_ID = '''||:random1||''' FOR '||REPLACE(DBMS_METADATA.get_ddl('INDEX', i.index_name, i.owner), CHR(10), ' '); -- cbo estimates index size based on explain plan for create index ddl EXECUTE IMMEDIATE sql_text; -- index owner and name do not fit on statement_id, thus using object_owner and object_name, using statement_id as processing state DELETE plan_table WHERE statement_id = :random1 AND (other_xml IS NULL OR NVL(DBMS_LOB.instr(other_xml, 'index_size'), 0) = 0); UPDATE plan_table SET object_owner = i.owner, object_name = i.index_name, statement_id = :random2 WHERE statement_id = :random1; EXCEPTION WHEN OTHERS THEN DBMS_OUTPUT.PUT_LINE(i.owner||'.'||i.index_name||': '||SQLERRM); DBMS_OUTPUT.PUT_LINE(DBMS_LOB.substr(sql_text)); END; END LOOP; END; / SET SERVEROUT OFF; WITH indexes AS ( SELECT pt.object_owner, pt.object_name, TO_NUMBER(EXTRACTVALUE(VALUE(d), '/info')) estimated_bytes FROM plan_table pt, TABLE(XMLSEQUENCE(EXTRACT(XMLTYPE(pt.other_xml), '/*/info'))) d WHERE pt.statement_id = :random2 AND pt.other_xml IS NOT NULL -- redundant AND DBMS_LOB.instr(pt.other_xml, 'index_size') > 0 -- redundant AND EXTRACTVALUE(VALUE(d), '/info/@type') = 'index_size' -- grab index_size type ), segments AS ( SELECT owner, segment_name, SUM(bytes) actual_bytes FROM dba_segments WHERE owner = NVL(UPPER(TRIM('&&owner.')), owner) -- optional schema owner name AND segment_name = NVL(UPPER(TRIM('&&index_name.')), segment_name) -- optional index name AND owner NOT IN &&exclusion_list. -- exclude non-application schemas AND owner NOT IN &&exclusion_list2. -- exclude more non-application schemas AND segment_type LIKE 'INDEX%' HAVING SUM(bytes) > POWER(2, 20) -- only indexes with actual size > 1 MB GROUP BY owner, segment_name ), list_bytes AS ( SELECT (s.actual_bytes - i.estimated_bytes) actual_minus_estimated, s.actual_bytes, i.estimated_bytes, i.object_owner, i.object_name FROM indexes i, segments s WHERE i.estimated_bytes > POWER(2, 20) -- only indexes with estimated size > 1 MB AND s.owner = i.object_owner AND s.segment_name = i.object_name ) SELECT ROUND(actual_minus_estimated / POWER(2, 20)) actual_minus_estimated, ROUND(actual_bytes / POWER(2, 20)) actual_mb, ROUND(estimated_bytes / POWER(2, 20)) estimated_mb, object_owner owner, object_name index_name FROM list_bytes WHERE actual_minus_estimated > POWER(2, 20) -- only differences > 1 MB ORDER BY 1 DESC, object_owner, object_name / DELETE plan_table WHERE statement_id IN (:random1, :random2); UNDEF owner table_name index_name exclusion_list exclusion_list2; SPO OFF;
Many tools offer Public APIs, which expose some functionality to other tools. SQLTXPLAIN contains also some Public APIs. They are provided by package SQLTXADMIN.SQLT$E. I would say the most relevant one is XTRACT_SQL_PUT_FILES_IN_DIR. This blog post is about this Public API and how it can be used by other tools to execute a SQLT XTRACT from PL/SQL instead of SQL*Plus.
Imagine a tool that deals with SQL statements, and with the click of a button it invokes SQLTXTRACT on a SQL of interest, and after a few minutes, most files created by SQLTXTRACT suddenly show on an OS pre-defined directory. Implementing this SQLT functionality on an external tool is extremely easy as you will see below.
Public API SQLTXADMIN.SQLT$E.XTRACT_SQL_PUT_FILES_IN_DIR inputs a SQL_ID and two other optional parameters: A tag to identify output files, and a directory name. Only SQL_ID parameter is mandatory, and the latter two are optional, but I recommend to pass values for all 3.
I used “Q1″ as a tag to be included in all output files. And I used staging directory “FROG_DIR” at the database layer, which points to “/home/oracle/frog” at the OS layer.
On sample below, I show how to use this Public API for a particular SQL_ID “8u0n7w1jug5dg”. I call this API from SQL*Plus, but keep in mind that if I were to call it from within a tool’s PL/SQL library, the method would be the same.
Another consideration is that Public API SQLTXADMIN.SQLT$E.XTRACT_SQL_PUT_FILES_IN_DIR may take several minutes to execute, so you may want to “queue” the request using a Task or a Job within the database. What is important here on this blog post is to explain and show how this Public API works.
Find below code snippet showing API Parameters. Notice this API is overloaded, so it may return the STATEMENT_ID or nothing. This STATEMENT_ID is the 5 digits number you see on each SQLT execution.
CREATE OR REPLACE PACKAGE &&tool_administer_schema..sqlt$e AUTHID CURRENT_USER AS /* $Header: 215187.1 sqcpkge.pks 12.1.03 2013/10/10 carlos.sierra mauro.pagano $ */ /*************************************************************************************/ /* ------------------------- * * public xtract_sql_put_files_in_dir * * executes sqlt xtract on a single sql then * puts all generated files into an os directory, * returning the sqlt statement id. * * ------------------------- */ FUNCTION xtract_sql_put_files_in_dir ( p_sql_id_or_hash_value IN VARCHAR2, p_out_file_identifier IN VARCHAR2 DEFAULT NULL, p_directory_name IN VARCHAR2 DEFAULT 'SQLT$STAGE' ) RETURN NUMBER; /* ------------------------- * * public xtract_sql_put_files_in_dir (overload) * * executes sqlt xtract on a single sql then * puts all generated files into an os directory. * * ------------------------- */ PROCEDURE xtract_sql_put_files_in_dir ( p_sql_id_or_hash_value IN VARCHAR2, p_out_file_identifier IN VARCHAR2 DEFAULT NULL, p_directory_name IN VARCHAR2 DEFAULT 'SQLT$STAGE' );
To implement Public API SQLTXADMIN.SQLT$E.XTRACT_SQL_PUT_FILES_IN_DIR on your tool, you need first to create and test a staging directory where the API will write files. This directory needs to be accessible to the “oracle” account, so I show below how to create sample directory “frog” while connected to the OS as “oracle”.
Since the API uses UTL_FILE, it is important that “oracle” can write into it, so be sure you test this UTL_FILE write after you create the directory and before you test Public API SQLTXADMIN.SQLT$E.XTRACT_SQL_PUT_FILES_IN_DIR.
Use code snippet provided below to test the UTL_FILE writing into this new staging OS directory.
DECLARE out_file_type UTL_FILE.file_type; BEGIN out_file_type := UTL_FILE.FOPEN ( location => 'FROG_DIR', filename => 'Test1.txt', open_mode => 'WB', max_linesize => 32767 ); END; /
On your tool, you can call this SQLT Public API from PL/SQL. You may want to use a Task or Job since the API may take several minutes to execute and you do not want the user to simply wait until SQLT completes.
Public API SQLTXADMIN.SQLT$E.XTRACT_SQL_PUT_FILES_IN_DIR is available for any 3rd party tool to use. If SQLT has been pre-installed on a system where your tool executes, then calling this API as shown above, will generate a set of SQLT files on a pre-defined staging OS directory.
If the system where you install your tool does not have SQLT pre-installed, your tool can direct its users to download and install SQLT out of My Oracle Support (MOS) under document 215187.1.
Once you generate all these SQLT XTRACT files into an OS staging directory, you may want to zip them, or make them visible to your tool user. If the latter, then show the “main” html report.
SQLT is an Oracle community tool hosted at Oracle MOS under 215187.1. This tool is not supported, but if you have a question or struggle while implementing this Public API, feel free to shoot me an email or post your question/concern on this blog.
Last night we had our get together for Oracle ACE members at Kscope14. Of course food and drinks were great, but a lot more important was the crowd. I am surprised to see how fast this ACE program is growing, and I believe the impact to our Oracle community is gaining momentum with it.
I will miss Kscope14 and I will submit some topics to Kscope15 soon. I already have a few new topics in mind, tailored to this “developers” Oracle Users Group at ODTUG.
Kscope14 has been great. I have attended a few sessions, and I enjoyed both the content and delivery. I am amazed to see how many topics are completely alien to me. And a new area of personal interest awakened on me yesterday: APEX! I would say: Once a Developer, always a Developer!
What I like most about Oracle Users Groups and Oracle ACE activities in general, is the opportunity to share with fellow techies, and learn from each other. Sessions are important, networking is important, but it is those short but intensive discussions on an aisle or booth that make an impact on me. It is refreshing to acknowledge there are many other passionate techies that would rather have an informal design discussion than watch the world cup!
I met Vikki from the Oracle ACE program and I finally put a face to a name. I actually met in person many other peers that I knew only by name or on the media. This Oracle ACE program is helping to build interrelations that translate into a better integrated Oracle community.
My kudos to the organizers of both Kscope14 and Oracle ACE. I take back home a great experience and a strong desired to continue contributing to our Oracle community.
While delivering a session at KSCOPE 14, I was asked a very good question: Can I create a SQL Plan Baseline for an Original SQL out of a Modified SQL? In other words, query Q1, which performs poorly, has SQL_ID1 and SQL_HANDLE1 with a poorly performing Execution Plan with Hash Value PHV1. With a small modification to this query, like adding a CBO Hint or removing one, we obtain query Q2, which performs well, and has SQL_ID2, SQL_HANDLE2 and PHV2. So what we want it to associate PHV2 to SQL_ID1. The questions is: how do we do that with an API?
Maria Colgan explains on this blog how to manually do exactly what is requested above. So the question becomes: can we do this manual procedure with an easy to use API?
Script coe_load_sql_baseline.sql below inputs SQL_ID1, SQL_ID2 and PHV2, which correspond to the Original SQL_ID, the Modified SQL_ID and the Plan Hash Value from the Modified SQL (it could have more than one plan); then it produces a SQL Plan Baseline for the Original SQL out of the Execution Plan of the Modified SQL. By doing this we can “fix” a Query performing poorly by attaching to it a good performing Execution Plan that was produced by making a small temporary change to the Original SQL, like adding or removing a small set of CBO Hints. So we can generate good Execution Plans without actually changing the application.
Script coe_load_sql_baseline.sql is provided inside SQLT (MOS 215187.1) under sqlt/utl directory. Use always the latest version. As of June 2014 the latest version is the one below. This script does not require to install SQLT.
SPO coe_load_sql_baseline.log; SET DEF ON TERM OFF ECHO ON FEED OFF VER OFF HEA ON LIN 2000 PAGES 100 LONG 8000000 LONGC 800000 TRIMS ON TI OFF TIMI OFF SERVEROUT ON SIZE 1000000 NUM 20 SQLP SQL>; SET SERVEROUT ON SIZE UNL; REM REM $Header: 215187.1 coe_load_sql_baseline.sql 22.214.171.124 2013/05/10 carlos.sierra $ REM REM Copyright (c) 2000-2013, Oracle Corporation. All rights reserved. REM REM AUTHOR REM email@example.com REM REM SCRIPT REM coe_load_sql_baseline.sql REM REM DESCRIPTION REM This script loads a plan from a modified SQL into the SQL REM Plan Baseline of the original SQL. REM If a good performing plan only reproduces with CBO Hints REM then you can load the plan of the modified version of the REM SQL into the SQL Plan Baseline of the orignal SQL. REM In other words, the original SQL can use the plan that was REM generated out of the SQL with hints. REM REM PRE-REQUISITES REM 1. Have in cache or AWR the text for the original SQL. REM 2. Have in cache the plan for the modified SQL REM (usually with hints). REM REM PARAMETERS REM 1. ORIGINAL_SQL_ID (required) REM 2. MODIFIED_SQL_ID (required) REM 3. PLAN_HASH_VALUE (required) REM REM EXECUTION REM 1. Connect into SQL*Plus as user with access to data dictionary REM and privileges to create SQL Plan Baselines. Do not use SYS. REM 2. Execute script coe_load_sql_baseline.sql passing first two REM parameters inline or until requested by script. REM 3. Provide plan hash value of the modified SQL when asked. REM REM EXAMPLE REM # sqlplus system REM SQL> START coe_load_sql_baseline.sql gnjy0mn4y9pbm b8f3mbkd8bkgh REM SQL> START coe_load_sql_baseline.sql; REM REM NOTES REM 1. This script works on 11g or higher. REM 2. For a similar script for 10g use coe_load_sql_profile.sql, REM which uses custom SQL Profiles instead of SQL Baselines. REM 3. For possible errors see coe_load_sql_baseline.log REM 4. Use a DBA user but not SYS. Do not connect as SYS as the staging REM table cannot be created in SYS schema and you will receive an error: REM ORA-19381: cannot create staging table in SYS schema REM SET TERM ON ECHO OFF; PRO PRO Parameter 1: PRO ORIGINAL_SQL_ID (required) PRO DEF original_sql_id = '&1'; PRO PRO Parameter 2: PRO MODIFIED_SQL_ID (required) PRO DEF modified_sql_id = '&2'; PRO WITH p AS ( SELECT DISTINCT plan_hash_value FROM gv$sql_plan WHERE sql_id = TRIM('&&modified_sql_id.') AND other_xml IS NOT NULL ), m AS ( SELECT plan_hash_value, SUM(elapsed_time)/SUM(executions) avg_et_secs FROM gv$sql WHERE sql_id = TRIM('&&modified_sql_id.') AND executions > 0 GROUP BY plan_hash_value ) SELECT p.plan_hash_value, ROUND(m.avg_et_secs/1e6, 3) avg_et_secs FROM p, m WHERE p.plan_hash_value = m.plan_hash_value ORDER BY avg_et_secs NULLS LAST; PRO PRO Parameter 3: PRO PLAN_HASH_VALUE (required) PRO DEF plan_hash_value = '&3'; PRO PRO Values passed to coe_load_sql_baseline: PRO ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ PRO ORIGINAL_SQL_ID: "&&original_sql_id." PRO MODIFIED_SQL_ID: "&&modified_sql_id." PRO PLAN_HASH_VALUE: "&&plan_hash_value." PRO WHENEVER SQLERROR EXIT SQL.SQLCODE; SET TERM OFF ECHO ON; -- trim parameters COL original_sql_id NEW_V original_sql_id FOR A30; COL modified_sql_id NEW_V modified_sql_id FOR A30; COL plan_hash_value NEW_V plan_hash_value FOR A30; SELECT TRIM('&&original_sql_id.') original_sql_id, TRIM('&&modified_sql_id.') modified_sql_id, TRIM('&&plan_hash_value.') plan_hash_value FROM DUAL; -- open log file SPO coe_load_sql_baseline_&&original_sql_id..log; GET coe_load_sql_baseline.log; . -- get user COL connected_user NEW_V connected_user FOR A30; SELECT USER connected_user FROM DUAL; VAR sql_text CLOB; VAR plan_name VARCHAR2(30); EXEC :sql_text := NULL; EXEC :plan_name := NULL; -- get sql_text from memory BEGIN SELECT REPLACE(sql_fulltext, CHR(00), ' ') INTO :sql_text FROM gv$sqlarea WHERE sql_id = TRIM('&&original_sql_id.') AND ROWNUM = 1; EXCEPTION WHEN OTHERS THEN DBMS_OUTPUT.PUT_LINE('getting original sql_text from memory: '||SQLERRM); :sql_text := NULL; END; / -- get sql_text from awr BEGIN IF :sql_text IS NULL OR NVL(DBMS_LOB.GETLENGTH(:sql_text), 0) = 0 THEN SELECT REPLACE(sql_text, CHR(00), ' ') INTO :sql_text FROM dba_hist_sqltext WHERE sql_id = TRIM('&&original_sql_id.') AND sql_text IS NOT NULL AND ROWNUM = 1; END IF; EXCEPTION WHEN OTHERS THEN DBMS_OUTPUT.PUT_LINE('getting original sql_text from awr: '||SQLERRM); :sql_text := NULL; END; / -- sql_text as found SELECT :sql_text FROM DUAL; -- check is sql_text for original sql is available SET TERM ON; BEGIN IF :sql_text IS NULL THEN RAISE_APPLICATION_ERROR(-20100, 'SQL_TEXT for original SQL_ID &&original_sql_id. was not found in memory (gv$sqlarea) or AWR (dba_hist_sqltext).'); END IF; END; / -- check phv is found DECLARE l_count NUMBER; BEGIN SELECT COUNT(*) INTO l_count FROM gv$sql WHERE sql_id = TRIM('&&modified_sql_id.') AND plan_hash_value = TO_NUMBER(TRIM('&&plan_hash_value.')); IF l_count = 0 THEN RAISE_APPLICATION_ERROR(-20110, 'PHV &&plan_hash_value. for modified SQL_ID &&modified_sql_id. was not be found in memory (gv$sql).'); END IF; END; / SET ECHO OFF; DECLARE plans NUMBER; description VARCHAR2(500); sys_sql_handle VARCHAR2(30); sys_plan_name VARCHAR2(30); BEGIN -- create sql_plan_baseline for original sql using plan from modified sql plans := DBMS_SPM.LOAD_PLANS_FROM_CURSOR_CACHE ( sql_id => TRIM('&&modified_sql_id.'), plan_hash_value => TO_NUMBER(TRIM('&&plan_hash_value.')), sql_text => :sql_text ); DBMS_OUTPUT.PUT_LINE('Plans Loaded: '||plans); -- find handle and plan_name for sql_plan_baseline just created SELECT sql_handle, plan_name INTO sys_sql_handle, sys_plan_name FROM dba_sql_plan_baselines WHERE creator = USER AND origin = 'MANUAL-LOAD' AND created = ( -- past 1 minute only SELECT MAX(created) max_created FROM dba_sql_plan_baselines WHERE creator = USER AND origin = 'MANUAL-LOAD' AND created > SYSDATE - (1/24/60)); DBMS_OUTPUT.PUT_LINE('sys_sql_handle: "'||sys_sql_handle||'"'); DBMS_OUTPUT.PUT_LINE('sys_plan_name: "'||sys_plan_name||'"'); -- update description of new sql_plan_baseline description := UPPER('original:'||TRIM('&&original_sql_id.')||' modified:'||TRIM('&&modified_sql_id.')||' phv:'||TRIM('&&plan_hash_value.')||' created by coe_load_sql_baseline.sql'); plans := DBMS_SPM.ALTER_SQL_PLAN_BASELINE ( sql_handle => sys_sql_handle, plan_name => sys_plan_name, attribute_name => 'description', attribute_value => description ); DBMS_OUTPUT.PUT_LINE(plans||' plan(s) modified description: "'||description||'"'); -- update plan_name of new sql_plan_baseline :plan_name := UPPER(TRIM('&&original_sql_id.')||'_'||TRIM('&&modified_sql_id.')); :plan_name := sys_plan_name; -- avoids ORA-38141: SQL plan baseline SQL_PLAN_64b0jqr2t1h3558b5ab4d does not exist IF :plan_name <> sys_plan_name THEN plans := DBMS_SPM.ALTER_SQL_PLAN_BASELINE ( sql_handle => sys_sql_handle, plan_name => sys_plan_name, attribute_name => 'plan_name', attribute_value => :plan_name ); DBMS_OUTPUT.PUT_LINE(plans||' plan(s) modified plan_name: "'||:plan_name||'"'); END IF; -- drop baseline staging table for original sql (if one exists) BEGIN DBMS_OUTPUT.PUT_LINE('dropping staging table "STGTAB_BASELINE_'||UPPER(TRIM('&&original_sql_id.'))||'"'); EXECUTE IMMEDIATE 'DROP TABLE STGTAB_BASELINE_'||UPPER(TRIM('&&original_sql_id.')); EXCEPTION WHEN OTHERS THEN DBMS_OUTPUT.PUT_LINE('staging table "STGTAB_BASELINE_'||UPPER(TRIM('&&original_sql_id.'))||'" did not exist'); END; -- create baseline staging table for original sql DBMS_OUTPUT.PUT_LINE('creating staging table "STGTAB_BASELINE_'||UPPER(TRIM('&&original_sql_id.'))||'"'); DBMS_SPM.CREATE_STGTAB_BASELINE ( table_name => 'STGTAB_BASELINE_'||UPPER(TRIM('&&original_sql_id.')), table_owner => '&&connected_user.' ); -- packs new baseline for original sql DBMS_OUTPUT.PUT_LINE('packaging new sql baseline into staging table "STGTAB_BASELINE_'||UPPER(TRIM('&&original_sql_id.'))||'"'); plans := DBMS_SPM.PACK_STGTAB_BASELINE ( table_name => 'STGTAB_BASELINE_'||UPPER(TRIM('&&original_sql_id.')), table_owner => '&&connected_user.', sql_handle => sys_sql_handle, plan_name => :plan_name ); DBMS_OUTPUT.PUT_LINE(plans||' pla(s) packaged'); END; / -- display details of new sql_plan_baseline SET ECHO ON; REM REM SQL Plan Baseline REM ~~~~~~~~~~~~~~~~~ REM SELECT signature, sql_handle, plan_name, enabled, accepted, fixed--, reproduced (avail on 126.96.36.199) FROM dba_sql_plan_baselines WHERE plan_name = :plan_name; SELECT description FROM dba_sql_plan_baselines WHERE plan_name = :plan_name; SET ECHO OFF; PRO PRO **************************************************************************** PRO * Enter &&connected_user. password to export staging table STGTAB_BASELINE_&&original_sql_id. PRO **************************************************************************** HOS exp &&connected_user. tables=&&connected_user..STGTAB_BASELINE_&&original_sql_id. file=STGTAB_BASELINE_&&original_sql_id..dmp statistics=NONE indexes=N constraints=N grants=N triggers=N PRO PRO If you need to implement this SQL Plan Baseline on a similar system, PRO import and unpack using these commands: PRO PRO imp &&connected_user. file=STGTAB_BASELINE_&&original_sql_id..dmp tables=STGTAB_BASELINE_&&original_sql_id. ignore=Y PRO PRO SET SERVEROUT ON;; PRO DECLARE PRO plans NUMBER;; PRO BEGIN PRO plans := DBMS_SPM.UNPACK_STGTAB_BASELINE('STGTAB_BASELINE_&&original_sql_id.', '&&connected_user.');; PRO DBMS_OUTPUT.PUT_LINE(plans||' plan(s) unpackaged');; PRO END;; PRO / PRO SPO OFF; HOS zip -m coe_load_sql_baseline_&&original_sql_id. coe_load_sql_baseline_&&original_sql_id..log STGTAB_BASELINE_&&original_sql_id..dmp coe_load_sql_baseline.log HOS zip -d coe_load_sql_baseline_&&original_sql_id. coe_load_sql_baseline.log WHENEVER SQLERROR CONTINUE; SET DEF ON TERM ON ECHO OFF FEED 6 VER ON HEA ON LIN 80 PAGES 14 LONG 80 LONGC 80 TRIMS OFF TI OFF TIMI OFF SERVEROUT OFF NUM 10 SQLP SQL>; SET SERVEROUT OFF; UNDEFINE 1 2 3 original_sql_id modified_sql_id plan_hash_value CL COL PRO PRO coe_load_sql_baseline completed.