It turns out, even xy-VSFilter and XySubFilter do not mangle colours if the video is native RGB regardless of the sub's YCbCr header. libass' docs were also updated to reflect this.