Advertisements

Hadoop Installation with Single DataNode on Ubuntu (VirtualBox)

Download link!

The above document contains :

01) Updating the Ubuntu repository.
02) Installing openjdk-6-jdk.
03) Installing openssh-server.
04) Downloading and installing Hadoop.
05) Configuring Hadoop enveronment.
07) Create a ssh key.
08) Format the name node
09) Start the namenode, datanode
10) Start the task tracker and job tracker.
11) Checking if Hadoop started correctly.

Advertisements

How to solve the issue caused by a poor estimate of cardinality in SQL Server.

Details:
The table statistics indicate that the tables involved have very few rows, but in reality, the row count is much higher.

Solution:
Updating statistics for the tables involved will improve performance.

Rebuild Table Statistics:

-- Execute the following query to rebuild the statistics held for the 
-- Proseware.Campaign and Proseware.CampaignResponse tables.
ALTER TABLE Proseware.Campaign REBUILD
GO
ALTER TABLE Proseware.CampaignResponse REBUILD;
GO

Lets re-run the query and take the execution plan.

Compare the Execution Plan:

Note that the estimated and actual row counts now match almost exactly. The query will execute faster than it did previously. The execution plan includes a suggestion for an index that would further improve query performance.

Create an Azure SQL Database

This document contains the steps for :
1. Creating a Free Trial Azure Subscription
2. Create an Azure SQL Database.
3. Configure Firewall Rules for your Azure SQL Database Server.
4. Installing and Connecting from a Client Tool
5. Setting up Microsoft Visual Studio.

Create_an_Azure_SQL_Database <download link>

Soft parsing of SQL statements was consuming significant database time.

Every query needs to be parsed before it got executed. If some queries are quite frequently used, a good design will be reuse the parsed query. That means, it will not parse the same query again (provided that we are using bind variables), rather will reuse the existing parsed query and then execute it. In an ideal world, execute should be more than the parse. More parsing requires more CPU.

Ideally when execute is more than parse, the figure will be positive. The ratio goes down if there are more parses than the execute. Then some of the queries are parsed, but less or never executed!.

Causes could be one of the followings:

  • There might be no “Prepared Statement caching” in java layer or in jdbc connection pool.

Solution:

Enable Statement Caching in Java:

// Enable statement caching

((OracleConnection)connection).setStatementCacheSize(x);

((OracleConnection)connection).setImplicitCachingEnabled(true);
  • There might be the case, before the execution of a query, the cursor was closed.
  • There might not be enough “session cached cursors” to hold the queries to reuse.

 

Fallback if you cannot change the application(java) to use statement caching :

session_cached_cursors = X
  • There might be the case where new queries were coming to play quite frequently.
  • There might be connection interruptions, this need to be investigate from network end.

Below SQL query will help to identify, the SQL’s which is parsed but not executed or less executed:

set linesize 200;
set pagesize 1000;
col sql_text format a40;
SELECT sq.sql_text, st.executions_total, st.parse_calls_total
, round( 100*(1-( st.parse_calls_total / st.executions_total )),2) execute_to_parse
, st.executions_delta, st.parse_calls_delta
, round( 100*(1-( st.parse_calls_delta / st.executions_delta )),2) delta_ratio
FROM DBA_HIST_SQLSTAT st
, DBA_HIST_SQLTEXT sq
, DBA_HIST_SNAPSHOT s
WHERE s.snap_id = st.snap_id
AND s.begin_interval_time >= to_date('2018-10-19 01:00 pm','YYYY-MM-DD HH:MI pm')
AND s.end_interval_time <= to_date('2018-10-19 03:00 pm','YYYY-MM-DD HH:MI pm') AND st.sql_id = sq.sql_id AND st.parsing_schema_name='<schema_owner>' AND st.executions_total !=0 AND st.executions_delta !=0 ORDER BY delta_ratio;

How to solve High ITL Waits for given segments

The segments ( table and it’s indexes)  are encountering High ITL (Interested Transaction List) waits, it is usually because of frequent inserts against blocks with insufficient space to support the demands of the concurrency requested of them.  So when a transaction occurs it marks the blocks that the transaction is ‘interested in’ modifying by placing an entry in the block’s ITL. As shown below by default the INITRANS value for a table is one  and for an index is two.

If there are insufficient free ITL slots then new ones will be taken in the free space reserved in the block. If this runs out and too many concurrent DML transactions are competing for the same data block then we observe contention against the following wait event – “enq: TX – allocate ITL entry”.

SQL> SELECT STATISTIC_NAME, VALUE FROM V$SEGMENT_STATISTICS WHERE OBJECT_NAME = '<Table or Index>'and VALUE > 200;

STATISTIC_NAME                   VALUE
--------------------------- ----------
logical reads                 25024144
db block changes               5300608
physical reads                  815272
physical writes                 260050
physical read requests          815272
physical write requests         234212
gc cr blocks received              281
gc current blocks received      698263
ITL waits                          575
row lock waits                   11068
space used                   108721232
space allocated              268435456

The database is affecting the “COMMIT/ROLLBACK”,is enq: TX – allocate ITL entry, – AWR report:

 

Enqueue Type (Request Reason)
Requests
Succ Gets
Failed Gets
Waits
TX-Transaction (allocate ITL entry)
4
4
0
2

Segments by ITL Waits (from AWR)

Owner
Tablespace Name
Object Name
Subobject Name
Obj. Type
ITL Waits
% of Capture
TEST
TEST_DATA
TEST_INDEX1
INDEX
3
100.00

There is a small section of predefined ITL (defined by the MAXTRANS parameter) in the block header, the ITL structure grows dynamically as concurrency occurs. If there is no space to handle the concurrency then waits occur. The primary cause of ITL waits is that free slots in the ITL structure in the block are not available. The lack of slots can be due to low setting of the INITRANS & MAXTRANS, which reserves minimal ITL slots initially and place a hard limit on the number of transactions that can have locks on a block respectively and/or, the block is so packed that there is no room for the ITL to grow Setting a high value of INITRANS will try to ensure that there are enough free slots in the ITL to hold all the concurrent transactions with a goal of minimal or no dynamic extension of the ITL. However, doing so also means that there is less space in the block for actual data, potentially increasing wasted space.

Another option is to try to make the data is less packed so that ITL can grow enough to accommodate the surges in ITL. This can be done by increasing PCTFREE, increasing FREELISTS and FREELIST GROUPS parameters for a table. This will make a block hold less data and more room for the ITL to grow. As a direct result of the reduction in packing, the table may experience fewer buffer busy wait events, and an increase in performance.

SOLUTION
The main solution to this issue is to increase the ITL capability of the table or index by re-creating it and altering the INITRANS or PCTFREE parameter to be able to handle more concurrent transactions. This in turn will help to reduce “enq: TX – allocate ITL entry” wait events.

To reduce enq: TX – allocate ITL entry” wait events, We need to follow the steps below:

A Combination of increasing both INITRANS and PCTFREE

1) Set INITRANS to 50 and  pct_free to 40

alter table <table_name> PCTFREE 20  INITRANS 50;

2) Re-organize the table using move (alter table <table_name> move;)

3) Then rebuild all the indexes of the table as below

alter index <index_name>  rebuild PCTFREE 20 INITRANS 50;

SELECT TABLE_NAME,INI_TRANS FROM DBA_TABLES WHERE TABLE_NAME IN ('<table_name>');
SELECT PCT_FREE FROM DBA_TABLES WHERE TABLE_NAME IN ('<table_name>');

Note :  Move the Table
Syntax: ALTER TABLE MOVE

Advantages:
1. This operation is fast especially when compared to shrink.
2. This can be performed using simple commands.

Drawbacks:
1. Table is not available for DML during the move operation.
2. Additional space is needed.
3. Indexes need to be rebuilt manually post move operation.

Additional Note: Alter table move ONLINE can be used only for :
-> index-organized tables and for nested table storage tables that are index organized. Using ONLINE clause, DML operations on the IOTs are allowed during rebuilding of the primary key index of the table.
-> For moving partitions and sub partitions online.

 

 

My 1st UDEMY course on Step-by-Step Instructions for Creating a Physical Standby Database (RMAN Active dupication)

https://www.udemy.com/oracle-data-guard-122/

screen_1

Kindly enroll this course and give me your valuable suggestion, so I can improve the content of the course accordingly:

This course is not detailing about what is Oracle Dataguard but it help you to build a Physical Standby as quick as possible for your further practices

What are the requirements?

Basic knowledge of Database Administration
Standard laptop / desktop for practicing course assignments
Windows / Mac / Linux PC, Minimum 8 GB RAM and 100 GB HDD Storage
Virtualization software VirtualBox – Create two machines
OEL 5.9 / 6.5 ISO image file to install Linux on Virtual Box

What am I going to get from this course?

Database creation using DBCA.
Preparing the Primary Database for Standby Database Creation.
Preparing the Standby system
Step-by-Step Instructions for Creating a Physical Standby Database (RMAN Active duplication)
Post-Creation Steps.
Bonus Topic: Oracle 12CR2: Creating PDBs from the Seed.

What is the target audience?

Anyone who would like to learn Oracle Data Guard Technology
On job people, who are looking to get a guidance to create a Physical Standby Database

Oracle 12CR2: Creating PDB from the Seed

• Create a PDB using SQL*Plus

• Delete a PDB using the DBCA

• Add the created Pluggable Database net service name in the tnsnames.ora file

DBCA error while creating CDB database.

Attempt to create a database in Oracle Restart environment fails and the following error

dbca_error_1.JPG

ORA-01017: invalid username/password; logon denied
. For details refer to "(:CLSN00107:)" in "/u01/app/grid/diag/crs/multisrv1/crs/trace/ohasd_oraagent_grid.trc".

CRS-2674: Start of 'ora.cdb1.db' on 'multisrv1' failed
ORA-01017: invalid username/password; logon denied
oracle.cluster.impl.crs.cops.CRSNativeResult.createException(CRSNativeResult.java:617)
oracle.cluster.impl.crs.cops.CRSNative.doStartResource(Native Method)
oracle.cluster.impl.crs.cops.CRSNative.genericStartResource(CRSNative.java:567)
oracle.cluster.impl.crs.cops.EntityOperations.startResource(EntityOperations.java:678)
oracle.cluster.impl.crs.CRSResourceImpl.start(CRSResourceImpl.java:883)
oracle.cluster.impl.crs.CRSResourceImpl.start(CRSResourceImpl.java:858)
oracle.cluster.impl.crs.CRSResourceImpl.start(CRSResourceImpl.java:846)
oracle.cluster.impl.common.SoftwareModuleImpl.start(SoftwareModuleImpl.java:525)
oracle.sysman.assistants.util.hasi.HADatabaseUtils.start(HADatabaseUtils.java:1476)
oracle.sysman.assistants.dbca.backend.PostDBCreationStep.executeImpl(PostDBCreationStep.java:1145)
oracle.sysman.assistants.util.step.BasicStep.execute(BasicStep.java:278)
oracle.sysman.assistants.util.step.Step.execute(Step.java:135)
oracle.sysman.assistants.util.step.StepContext$ModeRunner.run(StepContext.java:2941)
java.lang.Thread.run(Thread.java:745)
[Thread-291] [ 2018-05-01 10:46:38.078 GST ] [PostDBCreationStep.executeImpl:1156] Exception while Starting with HA Database Resource PRCR-1079 : Failed to start resource ora.cdb1.db
CRS-5017: The resource action "ora.cdb1.db start" encountered the following error:
ORA-01017: invalid username/password; logon denied
. For details refer to "(:CLSN00107:)" in "/u01/app/grid/diag/crs/multisrv1/crs/trace/ohasd_oraagent_grid.trc".

CRS-2674: Start of 'ora.cdb1.db' on 'multisrv1' failed
ORA-01017: invalid username/password; logon denied

 

Cause:

grid user is not under dba group.

[root@multisrv1 ~]# id grid
uid=54323(grid) gid=54321(oinstall) groups=54321(oinstall),492(vboxsf),54323(asmadmin),54324(asmdba)

Solution:

Add grid user to dba group and relink the oracle binary

 
[root@multisrv1 ~]# usermod -a -G dba grid
[root@multisrv1 ~]# id grid
uid=54323(grid) gid=54321(oinstall) groups=54321(oinstall),54322(dba),492(vboxsf),54323(asmadmin),54324(asmdba)

[oracle@multisrv1 bin]$ ./relink all
writing relink log to: /u01/app/oracle/product/12.2.0.1/db_1/install/relink.log
[oracle@multisrv1 bin]$