Tuesday, 22 August 2017

Educational Eclipse

Last night around midnight I laid in bed hoping that thousands of people on the other side of the world took the opportunity to experience the once in a life time experience that would be a total solar eclipse.

If you agree that a wonderful piece of prose can help recreate the experience, I recommend this literary non-fiction from 1982 by Annie Dillard.

Today I'd like to share three things I learned about the solar eclipse, considering I'm somewhat of an astronomy geek.

1) Shadow Bands

One of the amazing things about total solar eclipses is because they are relatively rare, there are still phenomena that science is trying to explain.

This video of an eclipse (which I recommend with a little sound) actually shows none of the classic imagery of a solar eclipse, but it captures so much of the awe.



And should I be lucky enough to witness one in future, I would have to ensure I experienced all that's available, even if it meant sticking up some big sheets of paper!

I think I might have a look like that of Katie Mack in the second photo, and probably the third.

2) Gravitational Lensing

In addition to the fact these eclipses are so awe inspiring, they do offer a bit of science, beyond helping calculate the dimensions of the solar system.

This example includes one of my favourite phenomena, more frequently associated with deep space imagery - gravitational lensing.

It turns out when the total eclipse happens, astronomers can take the opportunity to locate stars prominent enough to shine through. Compare distance between stars as observed at night, and astronomers can see how the sun's mass has nudged light, thereby confirming general relativity.
Stunning.

A concise article here with two great images, or google "Eddington 1919 eclipse"

3) Flight of the Concorde

All speeds involved with the eclipse are pretty wild, but back in the 70s, long before SpaceX started landing rockets on sea barges, there was a curious supersonic jet about to go commercial that took the public relations opportunity to fly faster than the speed of sound across Africa to accompany a 1973 eclipse.
Curious Droid - NASA chasing a shadow at 50k feet | Concorde 1973

I've been quite addicted to that channel recently, some great videos to watch during a lunchtime subway.

The first two I learned directly from Skeptoid Episode #584 by Brian Dunning. It's a great 12 minute podcast (now with ads) that I might describe as mixing science with anthropology. Very easy to pick a few interesting topics, then get hooked.

Did you see #Eclipse2017? Did you travel to get there?

Thursday, 17 August 2017

Revising jQuery with Oracle APEX

A few years ago I managed to bust out a book that explored my experiences using jQuery within Oracle APEX to solve various problems, many originally explored when building some "backend" apps for use on tablets.

For those who've read my book, thank you and I hope it was useful to you.
(I also hope you read the revised chapter 9 on calling PL/SQL from JavaScript.)
Please let me know what you thought of it with a review on Amazon.

The reason I bring this up is because I was asked on Twitter whether I would do a revision of my book for APEX 5.1.
I'm not sure if you've noticed, but my extra-curricular output has been down this year, despite my best intentions. That's not my answer, but it does weigh into what I think about the answer to that question.

Half of that book was written prior to APEX 5.0 being released. While a lot of the content is agnostic to the version of APEX you have, I must admit my techniques have evolved since the advent of the Page Designer.

That's not to say learning how to apply jQuery within APEX isn't useful. Having these fundamentals are still very much applicable, and useful to know when constructing dynamic actions; and even when defining CSS. But I think because it's much quicker to navigate through your dynamic actions in the Page Designer, I think there is less need to bundle logic within a JavaScript block.

I'm not sure revising this particular book for a later version of APEX would add value.

NASA Cassini
My video course had no particular theme and just explored various APEX techniques, hence the name. My book wasn't much different, and if I was going to do something again in future, it would possibly be along the same lines. Or something
even more applied. I do have ideas.

But I've been enjoying an awful lot of science in the time being. So much going on in space news!
It will be sad to see Cassini finish... Don't be surpised, I already have a post in mind for that amazing project.

In Memory Session State - Simple Example

Not too long ago I encountered an interesting behaviour with session state, turns out not long after Dick Dral posted a very similar article that I had not yet seen.

Recently I encountered an even easier scenario.

I had a bunch of columns where I wanted to use the same prefix across all headers, but I wanted to define the value once, referring to it many times. I'm betting on the fact someone may change their mind.

Interactive Report with pivoted data

So I created a hidden item called P222_PREFIX and gave it a static source of "Ch".

Page item source

If I substitute this item as part of the column heading for each column, it will use the value found session state - not the database value, but the value based on the defined source, placed only in memory for the duration of page rendering.

Column heading

So while the output may appear as desired after the page renders, if the region is partially refreshed by hitting "Go" on an interactive report or just triggering region refresh, the prefix will disappear because there is no value for P222_PREFIX in session state once the page finishes rendering.

Desired output

To fix the problem we add a computation on the hidden item to set the value, instead of using a static source. This ensures the value is in session state for current and later usage.

An alternative solution to the task at hand might be to use Shortcuts.

Monday, 7 August 2017

APEX Login Background Image Cover

For a while I've wanted to play with cover photos on login pages, and when Keegan asked a similar question on Twitter, I was curious enough to ultimately have a play.

Someone mentioned relevant a reference to CSS-Tricks Perfect Full Page Background Image, but after quickly finding the video Keegan must have screenshotted in her tweet, I realised us APEX developers need to use the following instead of 'html' as our selector.
.t-PageBody--login .t-Body

Plugin extraordinaire Daniel suggests
using 16:9 1920x1080 for standard UT use css media queries for different screen sizes with a pool of 2 / 3 images
The example I applied uses a CSS media query to not use a background image for smaller screens, as it may look too busy.
<style>
@media (min-width:400px)  { /* anything but mobile */
  .t-PageBody--login .t-Body {
    background: url(#IMAGE_PREFIX#cover_images/&P101_IMAGE.) no-repeat center center fixed;
    -webkit-background-size: cover;
    -moz-background-size: cover;
    -o-background-size: cover;
    background-size: cover;
  }
}
</style>
I defined this CSS within a region so I could apply a 'dev only' build option.

Style within Region
This also means the image location and file name are easily parameterised. So your images could be located anywhere, and you could programmatically decide which image to display. And/or use media queries to determine which image should display on the relevant device.

As an experiment, I wondered if I could rotate through a number of images, so each time someone visited the login page, they would see one image from a pool of many. So I defined an item with the following calculation:
'beauty'||floor(dbms_random.value(1,8))||'.jpg'

We can confirm an evenly distributed calculation by running that computation many times and counting the results.
select count(*), val from (
  select floor(dbms_random.value(1,8)) val
  from dual connect by level < 10000
) group by val

COUNT(*) VAL
1413  1
1445  6
1412  2
1420  5
1411  4
1415  3
1483  7
This would randomly select from a small suite of photos in the folder. I've used a selection I've collected from APOD.


And voila, an inspirational login page.

APEX Login with background cover image
I think an improvement would be to show a consistent image while attempting to login. A brief experiment suggests only running the computation when P101_IMAGE is null, and only clearing login fields (not the entire page cache) during page processing.

This could be the final bling you need after pimping your login page.

Don't forget, you can style the backend login, too.

Once again, thank you #orclapex community for making this a breeze.

Thursday, 20 July 2017

Sans Kscope

A few years ago I was lucky enough to attend Kscope15, and while I wasn't there this year, I have bizarre feeling of involvement.

The Buzz

I knew it was coming because the Twitter engine told me. I could have ignored Twitter for a week, or I could try garner some interesting information.

I could also live vicariously through all those Oracle fanatics sharing their Kscope experience on Twitter. I joke, but actually a lot of the gurus regularly share knowledge and interact in forums such as Twitter, Slack and OTN. If you're not involved in one of these feeds, you're probably missing out on a valueable source of contemporary tech information.

You may notice some of the most recent #Kscope17 tweets are from the after-party. It turns out this party is actually before the final 2 hour sessions on Thursday morning. Kudos to all those party animals that turned up to my jQuery deep dive in '15. 

Content

People share key slides. This gives us all a head's up for something photo worthy. Discussions start, interesting news and techniques get explored.

Key sessions were live streamed, and (I believe all) sessions will be available to ODTUG members as screen/voice recordings in September. For $99US a year, that's bargain training value. It's just setting aside an hour or so a week to knock off the relevant sessions.

Party

The Kscope party is amazing. I obviously wasn't there, but half a world away I experienced my own night out and I had a little epiphany, of sorts.

I was going to a music gig. No big name band, moreso a collaboration of local musicians performing a tribute to A Perfect Circle's Mer de Noms album. I know a few of you out there will have some clue as to what those words mean, or may like similar progressive rock. For me it was seeing a few mates perform an album I love live.

The gig was on at the same time as the Thursday morning live stream, but I wasn't missing this gig. It turned out to be an amazing gig. 

I've seen the singer (Dez) perform a number of times and he does a brilliant job emulating not only the original singer's voice, but also brings an amazing on stage presence. He smashed Tool's Aenima a few months ago at the same club, renowned for seasonal tribute gigs.
Another good friend was up there, pretending to know what to do with a six-string bass ;p, plus some other talented guys I've seen playing around before. And then out comes another dude up with an electric violin for a few songs!

Sea of Names - A Perfect Circle tribute

The sound was amazing, the small crowd was pumped, I had a lift to the gig so I actually enjoyed a few beers for a change. They did so well I had an early call of best gig ever, which is hard to judge & compare, but then I remembered seeing Roger Waters perform The Wall a few years ago and that will be hard to beat.

It ticked a lot of boxes, boys, well done. Encore.

Reflection

We're a long way from iconic music meccas like Seattle, but Perth kicks out some brilliant musicians, and I'm a grateful music lover.

For a few moments I thought about the Kscope attendees, having a fine time, letting off some steam after some solid days soaking up a lot of information.

I also thought about the dedication and skill required for people to learn a craft, then share it with others, be it music or code.

Congratulations ODTUG on creating an annual conference with such an impact.

Future

Many things afoot in this household at the moment, and if all goes smooth, we could find ourselves moved into a new home by the time abstracts close for Kscope18 in October.

That's the plan ;p

Science Friday: Collect rocks, plant flag

Every year that goes by firms my realisation that putting people on the moon was an absolutely stunning achievement.

Almost 50 years ago a massive team of engineers helped put 3 highly skilled men in orbit around that giant rock in our sky, that is stunningly distant, yes infinitesimally close, then land, then take off again.

While still doing the math by hand.

I think the best perspective of the distance in a scale overhead.

Ranging the moon
It's best behind a bunch of black, or a view from a telescope orbiting Mars., but consider it in light seconds.

Wiki

Then put some men in a hunk of metal utop a chemical missile rocket over a period of days.

Last year the twitter account @ReliveApollo11 recreated the days surrounding the landing, to help those of us too young to have experienced the timeline over the wireless as it happened.

Anyone can view it, and I highly recommend you try transport yourself to another time.

Blog title thanks to the 10c New York Times from that day #Apollo11

Friday, 23 June 2017

Removing Outliers using stddev()

A colleague asked me about removing outliers from reports and I knew just the function to give a go.

Some time ago I had to prove that some data we had conformed to a normal distribution. Remember those from high school?
A normal distrubition (bell curve)
That request was about 2007 and I ended up using stddev() and lpad() to produce a vertical shaped bell curve in SQL*Plus. That was pretty cool.

As for removing outliers, we can use this tool to exclude a small percentage of whatever data you have, so that it doesn't skew averages.

If you want to look at the code in context, I've created a test case on livesql.oracle.com
https://livesql.oracle.com/apex/livesql/file/content_E4749FJTJ7ZJY40CKU9HN3E0N.html

But as an inline reference, first I created a simple table, and populated it with a small set of random data between known range.
(I would like to add this is exactly what Tim Hall recently described here in regard to learning and asking questions.)

Importantly, I also inserted an outlier that was an order of magnitude higher than the rest.
My aim was to ignore this outlier in my SQL that calculated average.
create table outlier_demo  (a number, val number);

insert into outlier_demo  
select rownum -- we don't care  
  , round(dbms_random.value(20,50),1) -- some random numbers between 20 and 50  
from dual   
-- let's create 49 rows  
connect by level < 50

insert into outlier_demo values (0,500)

select * from outlier_demo;

A       VAL
---     ----
1       27.9
2       42.5
3       49.7
4       22.3
5       42.3
6       20.2
7       33.9
8       46.8
...
48      45.7
49      21.8
0       500
The average is higher than expected because of that outlier. I've included a column that calculates the average without the outlier, albiet hardcoded. Notice how close it is to the median? When considering usage statistics from APEX, I've often paid more attention to the median (most rows) than the average (affected by outliers).
select count(*) c   
   -- what's our average?   
  ,round(avg(val)) the_avg   
   -- what would the average be without the known outlier?   
  ,round(avg(case when val < 500 then val end)) no_outlier   
  ,round(stddev(val)) std   
   -- what's the median  
  ,round(median(val)) the_median  
from outlier_demo 

C    THE_AVG NO_OUTLIER    STD   THE_MEDIAN
---  ------- ----------  -----   ----------
50        46         37     66           36
We can take the same function and get the result for every row, that's one way we can use it to filter outlier data.
select a, val  
   -- what is 2 standard deviations away  
   -- provide result for each row  
   ,round(2*stddev(val) over (order by null)) local_std  
from outlier_demo
Re-written as a from clause, we can now filter any rows that have a value higher than x standard deviations from the norm.
select count(*) c   
  ,round(avg(val)) the_avg   
  ,round(avg(case when val < 500 then val end)) no_outlier   
  ,round(stddev(val)) new_stddev  
  ,round(avg(local_std)) filter  
from    
  (select a, val   
   -- what is 2 standard deviations away   
   -- provide result for each row   
   ,round(2*stddev(val) over (order by null)) local_std   
   from outlier_demo   
  )   
-- only average those values within two standard deviations   
where val < local_std;

C   THE_AVG NO_OUTLIER  NEW_STDDEV  FILTER
--  ------- ----------  ----------  ------
49       37        37           9      132
This could be re-written in a form that may be more recognisable to some.
select count(*) c
  ,avg(val) the_avg
  ,avg(case when val < 500 then val end) no_outlier
  ,stddev(val) std
from outlier_demo
where val < (select 2*stddev(val) from outlier_demo);
There are probably a number of variations on how to apply this solution, and possibly a few alternative methods.
It will probably depend on the SQL in your particular report.

This is not to say you should be ignoring outliers completely, but stddev() can help identify them where necessary. And it scales.

Thank Kate for this post.

ps - If you want to dive deeper, Jonathan Gennick (my patient book editor) has two in depth posts on the topic, with some diagrams to help understand the mathematical concepts.