Changing the NAT rule as you suggested made the trick!
Code:
# NAT
nat on $ext_if from !$ext_if to any -> ($ext_if)
I actually initially ruled this out as the root cause to the issue since I remember reading that it would only make a difference when the lease expires and I'm assigned a new external IP. Reloading the PF ruleset would set things straight in that case but didn't make any difference for me.
No doubt you saved me a weekend of hacking trying to sort the issue - thank you!!
// Magnus