Wednesday, 12 August 2009

Global/Local Oracle Community Networking

Recently while doing a little web surfing & blog investigating, I signed up to Oracle Mix and Oracle Community (my profile and profile).

It's sometimes difficult to get familiar with a new forum/website, and at the moment it seems a little overwhelming. However the idea for the two sites suggest good potential - I was wondering if anyone out there could provide thoughts on their experiences?
  1. Do you find it useful?
  2. Have you used it as an effective tool?
  3. What positives/negatives have you found?
  4. Do you have a preference?
  5. Are you happy to stick with an RSS blog feed?
g_input := 'welcome';

Monday, 10 August 2009

Subquery Factoring Clause - WITH

Through general experience I've found the WITH statement either underutilised or people haven't been aware of it's existence, although that seems to be changing - it has been around since 9i.

For me, two typical examples come to mind.
1) I was demonstrating recently an example of using SUBSTR & INSTR together to extract certain parts of a string. I built up the expressions step by step, then to test the example with a slightly different string, I would normally be forced to do a search and replace. Depending on the example, this can be annoying.
select 'halls head 6065'
,substr('halls head 6065',4,5) guess
,instr('halls head 6065',' ',-1,1) pos_of_space
,substr('halls head 6065',1,instr('halls head 6065',' ',-1,1)-1 )||'*' suburb
,'*'||substr('halls head 6065',instr('halls head 6065',' ',-1,1)+1 )||'*' postcode
from dual;
Instead I can put the string I want to play with in a simple WITH statement, and change the "column name" at will.
with test as (select 'halls head 6065' suburb from dual)
select suburb
,substr(suburb,4,5) guess
,instr(suburb,' ',-1,1) pos_of_space
,substr(suburb,1,instr(suburb,' ',-1,1)-1 )||'*' suburb
,'*'||substr(suburb,instr(suburb,' ',-1,1)+1 )||'*' postcode
from test;
Other times recently I have used the WITH statement to essentially encapsulate a unit of work. I had extracted a SQL statement from some PL/SQL and wanted to run it with my own restrictions, but leave the original untouched - this allowed me to test without worrying if I've disrupted the original join behaviour.
WITH original as ({copy of embedded sql})
select *
from original
where {my own restrictions}
These are ad hoc reasons, there is also a potential performance benefit from using the WITH clause.

I'm sure I've seen an example where using the WITH clause provided a performance benefit, but memory is fallible so I thought I'd see what the documentation had to say.
It says it will give the optimiser more choice about what to do, and that we can improve the query by using the WITH syntax.
After comparing explain plans from the example straight out of the documentation, I wasn't convinced. So I ran timings over different versions of the database - 1000 iterations each. I was surprised at the results (sounds like tabloid journalism, doesn't it?):

Without the clause
SELECT dname, SUM(sal) AS dept_total
FROM emp, dept
WHERE emp.deptno = dept.deptno
GROUP BY dname HAVING
SUM(sal) >
(
SELECT SUM(sal) * 1/3
FROM emp, dept
WHERE emp.deptno = dept.deptno
)
ORDER BY SUM(sal) DESC;

With the clause
WITH summary AS
(
SELECT dname, SUM(sal) AS dept_total
FROM emp, dept
WHERE emp.deptno = dept.deptno
GROUP BY dname
)
SELECT dname, dept_total
FROM summary
WHERE dept_total >
(
SELECT SUM(dept_total) * 1/3
FROM summary
)
ORDER BY dept_total DESC;
--9i
.3 secs
.0003 secs per iteration
2.9 secs
.0029 secs per iteration
--10g
1.1 secs
.0011 secs per iteration
4.06 secs
.00406 secs per iteration
--11g
1.17 secs
.00117 secs per iteration
7.46 secs
.00746 secs per iteration
Not only has it not performed better, its comparative performance has deteriorated as versions go up. So while we have removed duplicate code and enhanced the readability of our code, this example shows it's not necessarily more efficient.

It may just be a poor example to illustrate the performance benefit of this feature. It shows while a documented effect may be true in some circumstances, you must always test your particular scenario on your framework to get accurate comparisons. For further comment on this paradigm, I highly recommend reading this. Twice.

Monday, 3 August 2009

Interval Issues

Date functions have been ubiquitous within our database for years. For the most part date calculations are robust and can solve many problems.

Oracle even caters for a drift noted in the 8th century fixed in the 16th century - due to some issues with the dates chosen for Easter by the catholic church.
select date '1582-10-04' + 1 gregory from dual;

GREGORY
-------------------
15-10-1582 00:00:00
Let's take for instance one method of adding one month to a given day.
Here I add a month to the last day of August:
select add_months(last_day(date '2009-08-01'), 1) end_of_sept from dual;

END_OF_SEPT
-------------------
30-09-2009 00:00:00
And I safely get the last day of September (which has one less day).

Likewise, the documentation states:
For example, the MONTHS_BETWEEN function returns the number of months between two dates. The fractional portion of the result represents that portion of a 31-day month.
So these three expressions will return slightly different results
select months_between(last_day(date '2008-02-01'), (date '2008-02-01')) is_28_days
,months_between(last_day(date '2009-08-01'), (date '2009-08-01')) is_30_days
,months_between(last_day(date '2009-09-01'), (date '2009-09-01')) is_29_days
from dual;

IS_28_DAYS IS_30_DAYS IS_29_DAYS
---------- ---------- ----------
.903225806 .967741935 .935483871
However if you're working with intervals here is a little trap to watch out for.
select last_day(date '2009-08-01')
+ INTERVAL '1' MONTH end_of_sept
from dual;

+ INTERVAL '1' MONTH int
*
ERROR at line 2:
ORA-01839: date not valid for month specified
Interestingly if you subtract one month interval from the end of September you get 30th August. Similar behaviour occurs when subtracting from a leap February. It seems an interval of one month is considered as 31 days, but it can't allow properly for smaller months.
The same error will occur however if you attempt to subtract from a month such as July.
select last_day(date '2009-09-01')
- INTERVAL '1' MONTH end_of_aug
from dual;

END_OF_AUG
-------------------
30-08-2009 00:00:00
The 11g documentation states:
When interval calculations return a datetime value, the result must be an actual datetime value or the database returns an error...
SELECT TO_DATE('31-AUG-2004','DD-MON-YYYY') + TO_YMINTERVAL('0-1') FROM DUAL;
...
The first fails because adding one month to a 31-day month would result in September 31, which is not a valid date.
Personally I can't quite grasp why there should be a difference between interval arithmetic and functions such as months_between, but just be aware if you need to be pedantic with your dates.

Perhaps this behaviour with intervals will be modified in a future release?

Steven Feuerstein to visit Perth

If you haven't seen the AUSOUG broadcast, PL/SQL guru Steven Feuerstein will conduct a seminar in East Perth on 14th August 2009.

If not already, what better time to become a member of your local user group?

The agenda is currently available through the AUSOUG website here.

Hope to see you there!

Monday, 27 July 2009

Twitter -> Google Reader

I was playing with google seeing what keywords would return my blog, and I saw Chris Muir mention my presentation on his Twitter feed.

Now I have chosen not to fall into the Twitter trap, but there are some feeds I wouldn't mind following, Chris being one of them (Doctor Karl being another!)

For those uninitiated into RSS aggregators such as Google Reader, you can subscribe to these Twitterers & read what they have to say without needing to sign up to Twitter.


Kinda like reading all their mail, I guess... I can't tell you if there is some sort of privacy option within Twitter that controls this, perhaps someone with an account can?

I recently subscribed to one that had what I thought was a sensational idea - As part of the 40th anniversary, NASA arranged a Twitter feed for the moon landing, so you could read what was going on as if it was happening real time today. It gave a wonderful sense of time and progression for those of us not around when it happened.

I think it's nifty.

UNION ALL - Performance & Accuracy

Time for my first on topic Oracle titbit - UNION ALL

Take the following example. I had two mutually exclusive tables, fairly flat and uncomplicated. Both queries returned 17 rows.

I wanted all rows returned, regardless of uniqueness.

The original solution used UNION.
SELECT col_a, col_b, col_c
FROM table_1
UNION
SELECT col_a, col_b, col_c
FROM table_2

I knew in this case not only was this not optimal, it was also inaccurate. This case should have used UNION ALL
SELECT col_a, col_b, col_c
FROM table_1
UNION ALL
SELECT col_a, col_b, col_c
FROM table_2

Such a minor difference in syntax, but in this case ~30% faster and 34 rows returned.
Had two of these records had the same column values, less than 34 rows would have been returned. Sure, in other scenarios this may be accurate, but in others this may cause data integrity issues, throw out reports or lose data.

In terms of speed, my basic test harness iterated these two queries 50000 times and got the following results:
7.27 secs -- union
.0001454 secs per iteration

5.54 secs -- union all
.0001108 secs per iteration

Should this query only be executed once per day, probably not an issue in terms of speed. However if it were required thousands of times per minute then this would probably make an impact.

I think every little bit helps. Take the anology of Aussie Rules - Australia's favourite sport. Commentators say all those one percenters count, they can make the difference to the end result. Imagine taking 100 separate Oracle efficiency tips and bundling them within your software. Collectively, how much wait time will it ultimately save the user?

Thursday, 23 July 2009

July 2009 Branch Meeting

Thanks everyone for the great feedback from last night's presentation. I was happy with how it went, even though we were both a little rushed due to time constraints. For those who missed it, it's one for the catalogue...

My presentation is now available on my presentations page, and you can find the other Scott's presentation here.