I found
this, which matches your only EE message in Xorg.0.log, though I have no idea if this circumvention presented (manually building the missing swrast module) would have any affect whatsoever. You might post a follow up to it or reach out to Matthieu Herb directly.
A dmesg was not provided by the OP but my assumption is they were at -current, which would have been within a month or so of the tagging and release engineering of 5.4.