Things to keep in mind:
- Queing is on -outgoing- traffic only. There is no way to queue incoming traffic; by the time PF sees it, it has already arrived.
- The queue label you put on any pass rule applies to -state table- entries. In the case of your example, the inbound TCP traffic that matches that rule will have its state table entry flagged with the queue name "ssh".
- When -outbound- traffic associated with that state table entry is seen, PF will place it in its appropriate queue.
Specific to your question, I think you may be confusing direction with upload/download.
Consider that most uses of PF are on machines acting as routers. These machines have multiple NICs. PF doesn't know which NIC is used for what purpose. All it knows is traffic comes in on NICs, and goes out on NICs. You set up the queue assignments on any stateful connection, but you queue outbound traffic, whether heading out to the Internet, or heading out to your LAN.
Think of this example: A dual NIC simple router for a small network, and you, as an admin, wish to set up queues to control traffic. You can do so, both ways: outbound on your externally facing NIC, and outbound on your internally facing NIC. You can set up the state tables, however, based on any pass rule that allows state to be maintained.
Let us pretend you wish to limit incoming HTTP traffic on your local LAN, and decide to use a queue for this purpose. Outgoing HTTP traffic is any destined for the Internet, with a destination port 80. So the queue is described on the outbound pass rule on the external network. But your queue is set up on the internal network, as the traffic goes -out- from the router to workstations on your local LAN.