Wednesday, 11 January 2012

APEX performance issues with v()

One of the checks in the APEX Advisor is to determine if any instances of the v() function exist within your SQL.

This confirms one of the first APEX best practices I remember hearing, that you should use bind variables instead of v() within your SQL.
It makes sense, since v() is essentially making an interrogation on the session state table (wwv_flow_data).

Patrick Wolf talks more about performance with determinism here. Martin Giffy D'Souza summaries APEX variables nicely here.

In the comment thread Patrick also suggests the wonderful scalar subquery solution that works on all db versions
WHERE id = (SELECT v('MY_ITEM') FROM DUAL)

The trouble is, I think it's a difficult situation to test - how would you go about determining how many times Oracle decides to invoke the v() function, and for which parameters?

I came across this case was on a 10.2.0.4 db with Apex 3.1.2. I've added Groucho glasses to the query to protect the innocent/guilty.
It's also somewhat simplified. There were half a dozen instances of v(), but I've highlighted what turned out to be the offending line.

select * from a_6000_row_table p
where (  ...
or exists
  (select null
   from a_user_access_table aa
   where (aa.org_unit in (SELECT * FROM TABLE (InList_fn(a_pkg.genInList(p.year, p.org_unit))))
       or aa.org_unit in (SELECT * FROM TABLE (InList_fn(a_pkg.genDiffInList(p.year, p.id)))))
  and aa.inactive is null
  and aa.staff_id = v('LOCAL_USER_ID')
  and (  (aa.access_type in ('ABC','ABCXYZ'))
   or (   aa.access_type in ('DEF')
      and p.latest_status = 'YIPEE'))
  )

As it turns out line 9 wasn't alone in causing the query to take at least 5 seconds to return 1-10 of 6000 in an interactive report.
The first part of the where clause where it constructs a comma delimited list, converts it into a nested table, then interpreted as a table - also lends some form of hand in slowing down the query.

These function calls could probably be replaced with some form of WHERE EXISTS, but I think the original author had a steel plated encapsulation hat.

When first assessing this query, I thought the problem related to a stragg function in the column list which used a FOR-SELECT-LOOP, but then I spotted the obvious.
So I replaced all the v() references with bind variables, and all of a sudden the 5+ second query went sub one second.

I felt a little social, so I tweeted my little win, and Trent asked if I had more details, and I was curious to isolate the exact improvement as when I've briefly played with performance using v() functions on simple queries I came away unconvinced of any issues but happy to play it safe.

Some tests by themselves
  1. Removing the IN list functions got the query to around 0.7s
  2. Changing v() to a bind variable was around 1s
  3. Changing v() to scalar subquery also gave a 1s query.
  4. Setting entire query to use bind variables got down to about 0.5s
For those tuning experts out there who know much more about the Oracle mechanics than I do, I'd be curious to know if you can shed some light on how/why in particular these improvements came about.
For instance, what do you suppose the relationship between the IN list functions and the use of v() is?

For reference - I squeezed in a function call at line 10 that counted how many times it was called. It was always 6000 times, regardless of bind variable usage until I took out the call to the IN list, then it was 9 records. I didn't isolate the significance of that second number.

I remember when learning APEX it was difficult to determine when to use what syntax to refer to values in session state, so two lessons affirmed here today:
  1. Bind variables work! Limit the use of v() function calls to assignments and conditions in PL/SQL packages
  2. Scalar subqueries are just as awesome.
And I'll just add two more related points
  1. Never (never say never) use &ITEM. syntax in queries - I've got a post in the works on that issue.
  2. Don't forget about the nv() function, allowing you to compare the same datatype if you're dealing with numbers. You need to make your own dv function...
Scott

Monday, 9 January 2012

First events of 2012

Today is the true new year for many, and Sage Computing Services would like to announce it's first training event for the year open for enrolment.

JDeveloper Workshop
27th February to 2nd March
Perth CBD
Taught by Oracle Ace & Educator of the year - Penny Cookson
Workshop details

If you can't make that, keep the morning of the 29th Feb free. If things haven't changed on me over the new year, AUSOUG will be sending out details of a breakfast at the West Perth Oracle building by one of our favourites - Connor McDonald.

If you're interested in any of our courses, feel free to drop us a line and we can schedule a time frame that suits you.

Scott

Friday, 6 January 2012

Friday Fun: The SAGE Green Pig


The humble pig. Yep, it's been a while since the Perth conference, but for those of you who were there no doubt you'll remember the Sage Computing Services Green Piggies!




We've set up a tumblr containing a few of our favourite photos of piggies doing some interesting things.

If you happened to have pigged up a pig - take a photo of it doing something silly or useful and mail it through - Penny might decide it's G-rated enough to post on the page!


Scott

Wednesday, 4 January 2012

My APEX 4.2 wish list

UPDATE: Where to post your APEX feature requests.

I was going to start the new year with something a little more light-hearted, but this has been sitting as a draft for a while and I thought I better not hold onto it for too long.

As I'm sure many people do, while developing with the APEX tool, I occasionally come across improvements that I would love to see in a future version of Apex.

And, as I'm sure many people do, I keep meaning to note these little feature improvements down so I can provide feedback to the APEX development team.

Recently I managed to pull my finger out and actually compile a list - some training recently has highlighted some of these features so I made sure I took note when I could.

Some of them are really minor - aesthetic or workflow adjustments that would facilitate more fluid development. Others are probably a touch more complicated... I think most are for the benefit of the developer more than the end user.
Some probably need more thinking over, and some I may have gotten wrong and there isn't a problem anyway - feel free to correct me where I'm wrong. I'm also going to steer clear of any mobile development issues. Here we go...

Minor - little improvements I don't think would take much effort:

  1. Navigation bar entry subscriptions - currently when you attempt to define a subscription for navigation bar entries, the list of values displays the alt image text - not the navigation bar label.
    I don't have too many entries with this optional field completed, so it becomes annoying.
  2. Hide quick picks on read only - it would be nice to have a declarative feature that facilitates this - possibly turned on by default.
    Currently, if you conditionally set an item to read only, any quick picks are still present and clickable - hence updating the field.
  3. Display date/time in form as read only: I didn't investigate this one in depth - a last minute discovery in v3), but short of changing application date format to date/time - I had a strangely difficult time presenting a date column in a form page as read only with full date/time - I often received ERR-1079 Error in item post calculation computation.
    Feel free to direct me to an example of how this done, it wasn't a showstopper so I moved on.
Intermediate - slightly bigger, more involved improvements, but some of them may be easily achieved:
  1. Dynamic quick picks - this is a nice to have as we can fairly easily provide this using shortcuts (such weird terminology!)
    However since we have the option to define static quick picks for an item, why not be able to alternatively provide a query to dynamically supply the quick pick options, or associate an LOV shared component.
  2. API for page/item help - it is so often requested to facilitate the user to enter their own page level or even item level help. This requires it's own set of tables, template modification, supporting pages etc.
    If we had an API to the Apex core to update these values, all we'd need are pages to allow the user to read/update the help text so we can retain out-of-the-box functionality.
  3. View consolidating all comments - I originally thought this would be handy when playing around with the Forms migration tool. The annotations were great, but I thought how much handier would it be to be able to query one view to scan for relevant annotations.
    Ditto (and more-so) for component comments. We have an integrated page level comments that are actively visible to the developer, but as component comments can very easily be lost, ignored, forgotten about - I think it would be useful to run a query over any comments found in any of the application components.
  4. Greater usage of page alias - I think the use of a page alias instead of it's number should be more welcome within the development environment. This could apply to anywhere the link attributes are presented. This may also improve accuracy for users who bookmark pages within the application.
  5. Hierarchical validations - This was the first term I could think of. Currently we can tentatively apply conditions on validations based on failures of prior validations in the sequence using apex_application.g_inline_validation_error_cnt
    I think if we were able to define some form of hierarchy for validations, just like breadcrumbs or parent regions, this would make a lot of Apex developers very happy.
    I had a quick search for Apex wish lists before posting this, and Dan McGhan used the term validation chaining here.
  6. Validating file uploads - recently I needed to validate a file loaded using the File Browse widget. These validations included file size (we had a small limit, thanks to Report Queries) & mime type.
    I got there in the end, but it didn't seem very elegant. I'm not sure how easy it would be to provide settings for upper/lower bounds similar to those that numeric/date fields have.
  7. Minimum date setting - relax the format restriction on the date when using substitution syntax.
  8. Report Layout update process - currently if you need to update a report layout, you need to delete the layout record; re-upload the layout document; then re-associate the layout with any Report Queries.
    These are currently managed in separate areas of the Shared Components section, and if I remember correctly it takes 20+ clicks to achieve this process for one report. I jumped with glee when I found out recently I needed to update 50 layouts for 57 reports... not. It can't be too hard to re-jig these maintenance pages to allow a better workflow.
  9. Dictionary views for Report Layouts/Queries - are these the only components without associated dictionary views? I thought I'd be able to whip up a query to see which Report Queries used the same Report Layouts, but alas... I had to come up with a fresh technique.
    I got there in the end, and my update process for those 57 reports was impressively efficient, but both these improvements would make life easier.
  10. Form limits - my colleague wanted me to include extending the number of items per page from 100. She encountered the limit recently, but I'm not sure the context - 
  11. Multiple item copy - my colleague also would like the ability to copy more than one item at a time.
  12. Sequencing of next / previous buttons - when using the "<" and ">" buttons, ie - next / previous to scroll through Regions / Buttons / Branches etc, there seems to be no discernible order. If there is method to the madness, I haven't yet identified it. It should follow the sequence order. If that's what it's supposed to do, it's not evident in the behaviour.
Major - probably require some form of Apex core infrastructure changes:

  1. Build option enhancement - currently we assign a build option to a given component. This is great if it's new - but often there are minor tweaks on various attributes that we'd like to incorporate into a build. I find this even with template modifications. There are a few annoying workarounds, but I was trying to work out if it would be possible to incorporate a change to the Apex development environment.
    What came to mind (without thinking too hard about the underlying core structure - but I don't think it's too far fetched) is instead of (or in addition to) being able to allocate a build option to a component, it would be great to be able to have a select list perhaps on the side-bar, or near the "apply changes" button that would allow the developer to snapshot those changes against a given build.
    From another angle, when the developer is reviewing attributes for a component, they could view the current version (by default), or use the option to refresh the view for a given build option.
    That seems an decent scenario for the Apex developer - but as I said, I haven't yet put thought to the implications is would have on how Apex would manage this meta-data, and possibly more curious - how it would decide what to render.
    Perhaps the build option maintenance needs to be expanded to define chronology - and/or precedence.
  2. Saved IR reports retention - While I have not had too many problems in the environments I've been working in, it's definitely an area that needs attention, and the Apex product managers are aware of it. The basic premise of my issue with it is some life-cycle plans rotate through application IDs - this means there needs to be a method of transferring saved reports in production from one application to another. The export process leans towards the idea of migrating saved reports from one workspace to another - which to me seems limited as saved reports are typically not going to be in your dev/test workspaces.
  3. Expand XML size for Report Queries - this was the cause for my file upload validation request. If you would like to send an image to a BI publisher report via a report query, you are currently limited to a files size of approximately 22k. After a little chat with Tim Dexter, and a lot of trawling through OTN - it seems there is a limitation of 32k per "record" sent via XML.
    I can't remember where I put the sole forum reference, but after a quick search I found this 2008 posting by Marc Sewtz acknowledging the limitation and promising an update. It seems this limitation may affect charts also. I did attempt to use the print API instead as it seemed like a great alternative, however I came across what seems like some internal (Oracle end) scripting issues and I ran out of patience to persist.
  4. More declarative options for charts - I'm not a heavy chart user, but it seems the AnyChart infrastructure offers more customisations than the declarative nature of Apex allows. For instance - the graduated colouring on dial charts, and adding links for the same.
    Don't get me wrong, it's fantastic that it's possible to customise the XML sent to the engine, but I wonder how much work it might be to add context sensitive settings for the chosen chart style - even if it's done in a flexible manner similar to user-defined attribute pairs for Feedback.

    Added since initial posting from comments / forum post - limited to only those I like
  5. Jeff (2012/1/5) - New substitution strings such as APP_PAGE_ALIAS - and variables to return previous page details.
  6. Scott (2012/1/5) - New component in apex_application_install to accept clob that defines an exported Apex application
  7. Trent (2012/1/5) - Subscription module to team management
  8. Scott (2012/1/6) - Install supporting objects (stored as installation scripts) when running app install as script
  9. Wes (2012/1/7) - Filter objects in page definition by conditions; more options with search
  10. Andy  2012/1/7) - Declarative options for sub-totals in reports
  11. Matt (2012/1/12) - region/page level read only; row level cascading lov; extended APIs for tabular forms
  12. Greg (2012/1/15 - Documentation! - this is my favourite. I'm a huge fan of the db documentation, and the Apex documentation is still in its infancy. This includes item help - some are awesome and concise, some are not so much - typos in the help don't help the cause.

    Oh, and please revert back to 10g style doco - I hate the new 11g doco with the "expand all" option, and the local doco that attempts to hit the Oracle website - I need to configure an extension to stop that...
  13. Kofi (2012/1/13) - declarative popups
  14. Sas (2012/1/13) - alternate lov source for item
  15. Andy (2012/1/20) - Session expiry notificication
  16. Andy (2012/1/31) - (essentially) SQL based lists
Update: I gave up - replies went on for 6 months, 264 posts!

No doubt the APEX development team already have their 4.2 scope fairly set for an early 2012 release thanks to the jQuery mobile hype, but I'm sure there will be time to slot a few of these in if not target already - otherwise I can wait for 4.3 ;-)

Happy new year, everyone!

ScottWe

Tuesday, 20 December 2011

2012 AUSOUG Conference dates

Get your diaries out - the dates and locations for next year's conference series in Australia have been formally announced.


Melbourne
InSync12 Conference and Exhibition
August 21 and 22
The Sebel Conference Centre, Melbourne

This is a short tram ride from the heart of Melbourne, with stunning views of Albert Park - where they host the Grand Prix Formula 1.

Perth
Oracle with 20:20 Foresight Conference and Exhibition
October 29 and 30
Hilton Hotel, Perth

This is in the very centre of Perth, walking distance to bus, train, ferry & many other pubs & eateries.

The question is, will we see you in Australia next year?

Scott


Wednesday, 14 December 2011

Blog re-name & re-design (round 1)

Some of the more astute of you - or those that may have taken notice to the final "slide" of my Apex security presentation at the AUSOUG conference - may have noticed that I registered a new name for my blog.

There were a few reasons for this. Initially it took some time for me to decide on a name when I decided to start blogging, and in hindsight I thought it needed a re-jig. (my personal gmail account could probably do with the same adjustment)

I also thought that after getting into the groove of blogging, it was time to select a name that suited my particular style and removed the blogspot moniker.

So I've chosen the following:

grassroots-oracle.com

I've updated the about me page to describe this choice a little further. You may also notice I've got some other tabs across the top for common information you might like to view - using bloggers ability to create Pages for this very purpose. Feel free to check them out.

For those of you visiting my actual page as opposed to viewing through an RSS feed or aggregator, you may also notice the blog design has changed dramatically.

I've finally taken advantage of some of the new tools blogger provides. I don't think I've finished yet though. One reason - I'm on the road, it's really late, and I'm using my phone tether to work on the net.
The other reason - I like to sit on things for a while to see how a like it, and see if any other ideas come up. I'm also not quite happy with the colour scheme - I'm a little spoilt for choice and I wasn't quite ready to perform those finer adjustments.

I also think there may be some other ramifications from my template of choice - I notice my favicon has disappeared, and it seems I need to re-apply my syntax highlighting. Ahh, the life of a software developer...

And if you're wondering about the background image in the logo - I took that in my backyard using my smartphone, plus a little trickery - no software involved, except of course adding the text.

I hope you like the adjustments so far. If you have any tips or ideas, feel free to let me know.

Tuesday, 13 December 2011

A retrospective

Throughout our contemporary world there are many references to common denominations of time - and they all revolve around the same thing (pardon the pun).

Anniversaries of birthdays and various events are designated by a date in the year. These are also accompanied by decennaries, centennials, etc..

The moon, such an important part of our existence is also used to denote periods of time, albeit sometimes with less exactness.

So I thought why not use another cycle to mark an occasion, and look back over the years... that and I am basically a year late for the decennary.

Our planet experiences a solar cycle of about 11 years. It's still not completely understood why, and the various effects are still being studied - but after an extended dormancy our sun is starting to flare up (another pun!) and instigate various activity here on Earth.

My rambling aside - today marks 11 years since I got my first full-time job thanks to my university degree.

I never pro-actively sought after a job working with the Oracle product - in fact, my first job offer was in Sydney to work on the Collins class submarines using the Ada language - which was the teaching language of choice at ECU at the time.

So with much thought I declined that offer, deciding to stay in Perth - taking a job in a suburb many miles/kilometers from my home... using Oracle, and I hit the ground running.

Fortunately my first employer was often on the bleeding edge when it came to Oracle technology, so I learned a lot in a short amount of time.

With a little hesitation after almost 5 years I left my first employer so I could start exploring other industries (and perhaps work in town, instead of an industrial suburb well out of my way)

Wind forward to today, and I've touched on Forms, Reports, Designer, Portal, Discoverer, EBS, mod plsql, and a seemingly longish gap to Application Express.

I think more importantly, however, I've been active in my local user group. I think when it comes to advancing my career, this has been one of the most important aspects.

Participating in user group events allowed me to meet & greet others in my community, in hindsight allowing me to maintain consistent contact with my future employer (Sage). Ultimately I started presenting papers of my own, rewarding me with a best paper award at this year's AUSOUG conference.

It's also opened a number of other doors, almost like a positive feedback loop. I've reviewed a number of new Oracle books from Packt Publishing; and I've been a technical reviewer in a number of interesting Oracle books from Apress. This is another part of the feedback loop, since I'm learning & reasserting skills while reviewing books I'd want to read anyway.

As for the future? Well, from a technology perspective I think in due time my lovely employer Penny would like me to learn JDeveloper, but in the nearer future I'm more than happy to continue exploring and improving my Oracle Apex skills.

I will certainly be continuing my extra curricular activities. Amongst other tasks, soon I hope to co-author some articles with Penny; I have some more ideas for fresh presentations using the Prezi tool - limiting death by powerpoint!; I hope to finally redesign this blog and "officially" launch the new name, but that may be competing with our yearn to translate our Sage website into Apex - which will offer many new opportunities. I've just got to make sure I don't destroy the wonderful SEO work Chris Muir has put into the static HTML version.

Ultimately, it would be great to start participating in some of the bigger events around the world - UKOUG & ODTUG's Kaleidoscope, for example. It's just a little shame we're here in the most isolated capital city in the world... but it's worth it - I love living here ;-)