First, I am not a pf expert.
Try the following example and see if it works, then change the non functional parts, and/or tight the rules as sutable to you.
Code:
ext_if="rl0" #
int_if="rl1" #
lan="192.168.10.0/24"
set loginterface $ext_if
table <user1_ips> { 192.168.10.2, 192.168.10.5 }
table <user2_ips> { 192.168.10.3 }
scrub in all
# NAT
nat on $ext_if from $lan to !$lan -> $ext_if
# IN
altq on $int_if cbq bandwidth 99Mb queue { inet_in, default_in }
queue inet_in bandwidth 512Kb { user1_in, user2_in }
queue user1_in bandwidth 50% cbq(red, borrow)
queue user2_in bandwidth 50% cbq(red, borrow)
queue default_in bandwidth 99% cbq(default)
# OUT
altq on $ext_if cbq bandwidth 99Mb queue { inet_out, default_out }
queue inet_out bandwidth 256Kb { user1_out, user2_out }
queue user1_out bandwidth 50% cbq(red, borrow)
queue user2_out bandwidth 50% cbq(red, borrow)
queue default_out bandwidth 99% cbq(default)
block in on $ext_if all
block out on $ext_if all
# TAGGING on users
pass in on $int_if from $lan
pass in on $int_if from <user1_ips> to any tag USER1
pass in on $int_if from <user2_ips> to any tag USER2
# Queues
# IN
pass out on $int_if from any to $lan queue default_in
pass out on $int_if from any to <user1_ips> queue user1_in
pass out on $int_if from any to <user2_ips> queue user2_in
# OUT
pass out on $ext_if from ($ext_if) to any queue default_out
pass out on $ext_if from ($ext_if) to any tagged USER1 queue user1_out
pass out on $ext_if from ($ext_if) to any tagged USER2 queue user2_out
This was not tested, I hope it can help you towards the right path.