I forgot to convert the video's colorspace, it uses 4:4:4 chroma sampling, which is too much and not many decoders supports it. So I convert its colorspace to 4:2:0 chroma sampling, the same as every video in the world. It should work now.
AHA! I think I know the reason... I forgot to convert the video's colorspace, it uses 4:4:4 chroma