Friday, 12 January 2018

Data driven APEX icons

We have an application written with a heavily customised Theme 25 built for 10" tablets, and we feel the Universal Theme justifies the move, in part because of the surrounding ecosystem. Check out this forum discussion on the topic.

We've been looking through the packaged applications for applied ideas, and using the Universal Theme sample application as a component reference.

I came across a requirement where we had a list of items that indicated completion level out of 100.
Then I thought about a group of pie icons I saw in Dick Dral's Font APEX icon reference, and knew what I wanted to do.

All I needed to do was round a bunch of numbers to the nearest 5.

To solve such as problem, I usually start by giving myself a bunch of numbers to play with
select rownum rn from dual connect by level <= 100

Then defining that as a WITH statement, so I can refer to the derived column as often as I like.
with nbrs as (
  select rownum rn from dual connect by level <= 100
)
select rn
 , floor(rn/100 * 20) / 20*100 rnd
 ,'fa fa-pie-chart-'||floor(rn/100 * 20) / 20*100 icon
from nbrs
where mod(rn,5)=0
And I used a divide/multiply by 20 math trick to do the rounding.

We can then feed the result as the relevant icon into whatever region template we need.


I like pie.

Simple, yet effective.

Tuesday, 9 January 2018

Modify OracleJET Property at Runtime in APEX

OracleJET has attributes galore, but some are are (not yet) available to change at design time, so JavaScript code can be added to the chart attributes to set relevant attributes.
function(options) {
options.styleDefaults.threeDEffect = "on";

return options;
}
See my previous post about modifying these attributes on render.
We can also do this at runtime, perhaps as response to a button click, such as the 2D/3D button in the cookbook.

First, set a static ID on the chart, possibly one of the most common "advanced" properties I use.

Static ID property available on many components

Use this ID in the browser console at runtime to see what JSON was generated for the chart definition.
Any missing properties will use the default specified in the documentation.
apex.widget.jet.getChartJSON('p95_skew')

Browser Console results

The documentation on these attributes is thorough, but I'd love some examples to help keep me moving.
http://www.oracle.com/webfolder/technetwork/jet/jsdocs/oj.ojChart.html#styleDefaults.threeDEffect

We can modify the orientation property by supplying a name/value pair as a JSON string.
$("#p95_skew_jet").ojChart({'orientation': 'horizontal'});
Note the selector has is the static ID with a suffix: $("the_static_id"+"_jet")

It took a little while to find the correct punctuation for the nested properties, so this is really one of those blog posts I created so I don't forget. You've seen Alex's blog by-line, right?
$("#p95_skew_jet").ojChart({'styleDefaults':{'threeDEffect': 'off'}});
Not all runtime tweaks behaved as expected, however. The following property behaves as expected when setting on render, but at runtime it squishes the width of the entire chart.
$("#jet1").ojChart({'styleDefaults':{'barGapRatio': 0.2}});

Refreshing the chart region afterwards did not help in this case.
While looking for answers, I came across this post from Riaz describing similar customisations.

OracleJET JavaScript Customisation in APEX

I've finally got some regular hands on a 5.1 instance, and the shiniest tool in the box for me is OracleJET.

Some months ago I spent a few days learning about OracleJET and the knockout framework with Chris Muir. I doubt I'd ever get to that nitty gritty, but it sure is handy to know some of the finer details now that I'm using them in APEX.

I wanted to have a play with the funnel chart. I took an existing query, and quickly busted out a working chart.

Upon playing with the sample chart in the OracleJET Cookbook, I decided I wanted the 3D option.
In the case of this chart style, I think the subtle effect made a big difference.

The Cookbook is a great guide to what you can play with
I couldn't find the relevant attribute defined declaratively in APEX, but that's fine - I know all of them aren't mapped, and we have a special JavaScript section in the chart attributes to help us customise the content. This expands upon attribute selections, not replacing them like the custom XML did for AnyCharts.

The attribute help in the Page Designer is a good start, and always worth checking when playing with a new field.

Page Designer Attribute Help

I noticed the Sample Charts application did the same thing, but I figured I could take this sample from the help, combined with information from the impressive JET documentation, and try work out the JavaScript myself.

Not all properties are mapped to APEX attributes

This is what I came up with.
function(options) {
  options.styleDefaults.threeDEffect = "on";
  
  return options;
}

Only to find the customisation from the sample application looked same same, but different.
function( options ){
    options.styleDefaults = {
        threeDEffect: "on" 
    };
    return options;
}
Turns out both are effective, once again demonstrating there's always a few different ways of doing the same thing in JavaScript.
I suspect the first option honours the object notation mentioned in the documentation, while the second assigns the value as a JSON name/value pair. I've seen similar behaviour in jQuery:

$('#item').css('color','blue')
vs
$('#item').css({'color':'blue'})

You can find other examples in your workspace with this query on the dictionary view.
select application_id, page_id, page_name, region_name, chart_type
 ,javascript_code
from apex_application_page_charts 
where javascript_code is not null;
You may also enjoy changing series colours with similar treatment by Colin Archer of Explorer UK.
http://www.explorer.uk.com/customising-chart-colours-apex-5-1/
The German APEX community also has a thorough rundown of JET charting in 5.1
https://apex.oracle.com/pls/apex/germancommunities/apexcommunity/tipp/5841/index-en.html

In my next post I'll explore how to play with these attributes at runtime, not just on render.

Thursday, 21 December 2017

ORDS Cross Origin Complaint

A few years ago while upgrading to APEX 5.0, we had a few issues when upgrading ORDS.

My loose understanding is that from about ORDS 3.0.3, it started enforcing some security policy regarding cross-origin requests. The big browsers handle this differently, for instance Chrome returns 403 Forbidden and won't let you log in.

Cunning Chrome Cross Origin Complaint
Amusingly, Edge let me in, even after advising me otherwise. No doubt there is minutiae I do not yet understand.

HTTP403: FORBIDDEN - The server understood the request, but is refusing to fulfill it. (XHR)POST - https://redacted.com.au/vbs/wwv_flow.ajax

It seems depending on what you're trying to accomplish, and your middle tier framework, your problem/solution may be different.

I found these threads brimming with leads
https://community.oracle.com/thread/4042054
https://community.oracle.com/thread/3971878
And while writing this post I found one that essentially described our solution
https://community.oracle.com/message/1409513

Now most discussions involving proxies are word soup to me, especially when they're described as reverse proxies.
But I think this experienced helped me wrap my head around it. A little.

For us, outside requests got remapped to an internal domain. This effectively meant the domain looked different, and somewhere along the line some bit of software drew the line and said this was a security threat.

However, there is a setting that allows the Proxy Host name to be preserved.
It's aptly named ProxyPreserveHost, or preserveHostHeader, depending on your middle tier kit.
And it's described well here
https://www.ctrl.blog/entry/how-to-httpd-reverseproxy-hostheader
No complaints since.

So for anyone migrating to ORDS 3.0.3+, perhaps don't lay blame on ORDS for your problems.
There may be some proxy voodoo your network engineer can fix in a jiffy, with that golden screwdriver. Thanks, Harry.

ORDS web services returning BLOBs

When it comes to deliving blobs from the database, I'm sure many of us have used, or came across a procedure that look like the one described here.

This sample includes some commentary on how the surrounding infrastructure should look, but it's a little out of date.
create or replace procedure get_image(p_id  IN  images.id%TYPE) IS
  l_mime        images.image_type%TYPE;
  l_length      NUMBER;
  l_lob         BLOB;
-- This procedure needs
  -- Grant to apex_public_user
  -- Public synonym, so not reliant on #OWNER#
  -- Function wwv_flow_epg_include_mod_local needs to include this procedure name
  -- Test rendering using call such as
  -- http://domain.com.au/pls/apex/GET_IMAGE?p_id=2
  -- If not found, check case, synonym
  -- If no permission, check execute priv, presence in wwv_flow_epg_include_mod_local
BEGIN
  IF p_id IS NOT NULL THEN
    SELECT image_type, the_blob, DBMS_LOB.GETLENGTH(the_blob)
    INTO   l_mime, l_lob, l_length
    FROM   images
    WHERE  id = p_id;

    OWA_UTIL.MIME_HEADER(COALESCE(l_mime, 'application/octet'), FALSE);
    HTP.P ('Content-length: '||l_length);
    OWA_UTIL.HTTP_HEADER_CLOSE;
    WPG_DOCLOAD.DOWNLOAD_FILE(l_lob);
  END IF;
END get_image;
There are alternative methods available, so you can make ORDS more inherently secure, as descibed by Kris Rice. In fact, this was the reason I needed to do something. Regression testing with a newer version of ORDS uncovered a difference with requestValidationFunction parameter.

For instance, you could use an application process, thanks to Joel Kallman for yet another solution ;p.

We've previously delivered other images using web services, thanking Kris again for the inspiration. The web service is a basic query on the images table, and it also means referencing the image within the HTML is easy
<img src="/ords/rest/image/fetch/3141592" />

Often this would translate to an expression like this within your report
<img src="/ords/rest/image/fetch/#IMAGE_ID#" />

Or perhaps
<img src="#YOUR_IMAGE_PATH##IMAGE_ID#" />

I recently upgraded SQL Developer to a relatively shiny 17.3, so I could use the fancy REST navigator in the connections window.

I've also been meaning to note down the process, and typical values, since I still only understand web services at a relatively basic level.
(secret: the real reason we write blogs is so we can google ourselves later.)

I new based on existing web services that the following values were required, so I just had to match these to the steps in the create wizard.

Module: image
URI Prefix: /image/
URI Pattern: fetch/{id}

The module being a group of templates, which are perhaps difference sources of images -> image table, user table, product table. Then a handler is created to get or post data.

The first step is a good start. I've always had trouble working out what value goes where in the URL equation, so it's great to see the translation in the first step.

Step 1 - Define Module

Step 2 finalises the URL by defining how the identifier for the database record will be provided.

Step 2 - Define Template

And we're at the summary already, noting we can create web services without initially publishing them for use (from step 1).

Step 3 - Summary

We can flip over to the SQL tab to show all that SQL Developer is doing behind the scenes - calling supplied APIs.

Step 3b - Review SQL

Now the web service should be visible in the SQL Developer Connections navigator.

SQL Developer UI

Righ click on the template to create a handler, in this case to GET the image.

Step 4 - Add Handler

All the source types return some form of text, except Media Resource, which delivers a binary representation of our blob.

Handler Type

And the handler is just a SQL statement fetching the blob from the table, filtering with a bind variable mapped to the {id} named in the URI pattern.

Handler SQL

The query in the image:
select image_type as mimetype
,the_blob
from images
where id = :id
All pretty simple so far, but we had another similar procedure that delivered binary content using bfilename(). ie - files in the file system, resolved via an Oracle Directory.
I figured I could still write a query that delivered this using a table function:
select mime_type, document
from table(get_job_doc(:code));
Where get_job_doc returns an object type. The function doesn't need to be pipelined, because it only returns the one row. I describe the relevant pseudo-code in this forum post.
It works, but I had trouble adding content-disposition in the header, so .msg files don't download well. I bet there is some tidy ORDS feature/solution I'm yet to discover.

In the end, one procedure was replaced with a web service, another with an application process. Both more secure options that don't require custom PL/SQL to be available via the URL. A whitelist is safer when there's no need for one.

In the era of cloud, web services are king.
Now we just need to wait for ORDS & APEX to talk the same data set when it comes to manipulating these services.

Wednesday, 20 December 2017

On Controlling Use of Minified Code in APEX

Today I encounted an issue during regression testing for 5.1. An older application that doesn't use the Universal Theme, and hasn't really been touched since 4.x, complained about the SkillBuilders modal page plugin.

The solution seemed pretty clear upon reading through this forum post - just update the relevant jquery.colorbox.js file to the latest version, no problem.

I just updated the minified version on my dev instance, jquery.colorbox-min.js, but saw no effect. After checking the code in my relevant plugin package (since all our plugin PL/SQL goes into packages), I saw it wasn't actually using the minified version, and it got me thinking.
apex_javascript.add_library(
      -- p_name      => 'jquery.colorbox-min',
      p_name      => 'jquery.colorbox',
      p_directory => p_plugin.file_prefix,
      p_version   => NULL
   );
Quite some time ago I remember learning about the #MIN# substitution string. The idea is you include files using my.file#MIN#.js, and it only used the minified code when you weren't in debug mode. (That double negative is just for Eddie.)

Item help when adding File URLs
My initial thought was to add it myself, based on the value of debug.
apex_javascript.add_library(
      p_name      => 'jquery.colorbox'||case when v('DEBUG') = 'NO' then '-min' end,      
      p_directory => p_plugin.file_prefix,
      p_version   => NULL
   );
Only to quickly realise the apex_javascript.add_library API has a dedicated parameter for such a task, though it expects dot notation in the filename, not a dash: my.file.min.js
apex_javascript.add_library(
      p_name      => 'jquery.colorbox',
      p_check_to_add_minified => true,
      p_directory => p_plugin.file_prefix,
      p_version   => NULL
   );
What surprised me though was looking through the other plugins. I couldn't find any in my current data source that used the p_check_to_add_minified parameter, they all just expected its use.

Do plugin developers not bother with p_check_to_add_minified?

Perhaps the newer plugins at apex.world use the parameter, if so I'll stand corrected.

Or perhaps there's little value for its use in our data driven pages, once the plugin is up and running. Any APEX plugin authors care to chime in?

Friday, 8 December 2017

Friday Fun SQL Lesson - union all

Our office kitchen is unavailable this Friday, so the call was put out for pub lunch.

After a couple of replies I decided to enter my reply, in full nerd mode.
select * from people_coming_to_lunch;

People
--------
Kate
Scott
Karolina

3 rows selected.
And of course one of the other SQL geeks (name redacted) replied to extend the data set.
select * from people_coming_to_lunch
union 
select 'Shanequa'
from dual;
And I couldn't help myself. I had to play the performance card and suggest that UNION ALL would be the appropriate usage, and should be the default you type out. Always. Until you decide otherwise.

That's because sorts are expensive. And a UNION will need sorting to check for duplicates.

That sort of all the rows isn't necessary
And it will sort the data set even if there is a unique key on the data.
create table people_coming_to_lunch (people varchar2(30));
insert into people_coming_to_lunch values ('Scott');
insert into people_coming_to_lunch values ('Kate');
insert into people_coming_to_lunch values ('Karolina');

create unique index lunch_people on people_coming_to_lunch(people);

select * from people_coming_to_lunch
union all
select 'Shanequa' from dual
By using UNION ALL instead of UNION, you're telling the database not to even bother sorting the set to eliminate any potential duplicates, since your advanced human brain knows there will be no duplicates.

With only a few rows, the timing of sheer throughput is barely noticable.
iterations:1000
     0.30 secs (.0003 secs per iteration)  -- UNION
     0.25 secs (.00025 secs per iteration) -- UNION ALL
 
iterations:10000
     1.72 secs (.000172 secs per iteration)
     1.09 secs (.000109 secs per iteration)
 
iterations:50000
    10.94 secs (.0002188 secs per iteration)
     8.48 secs (.0001696 secs per iteration)
So I turned it up a notch and added about 5000 rows to the table.
insert into people_coming_to_lunch
select table_name from all_tables;
 
5000 rows inserted
Here's the explain plan without the sort.

That's one less chunk of 5000 rows to process

Now the differences in performance stand out.
iterations:1000
     6.79 secs (.00679 secs per iteration) -- UNION
     2.85 secs (.00285 secs per iteration) -- UNION ALL
 
iterations:5000
    42.91 secs (.008582 secs per iteration)
    19.89 secs (.003978 secs per iteration)
 
iterations:5000
    31.70 secs (.00634 secs per iteration)
    22.83 secs (.004566 secs per iteration)
 
iterations:5000
    30.75 secs (.00615 secs per iteration)
    16.76 secs (.003352 secs per iteration)
Upto twice as long for the same statement?
No thanks, not when I could just type 4 extra characters to get an easy performance win.

Turns out this topic formed my first technical post. Back in 2009, after almost 10 years of using SQL, that was the first thing I blogged about. How about that.