We are also experiencing this issue since we began Geocaching. We are using a Xiaomi Mi 9T Pro and our currrent Geocaching App version is 9.21.0.
For us, even Choose from Library doesn't work properly with pictures taken using the stock Camera app, and they appear rotated.
However, we quickly found a workaround back in the days:
You can edit the photo you've taken in your Gallery app and crop it (you don't have to crop much, just a tiny bit) and then attach the cropped image.
This cropped image then has the correct orientation.
In the attachments, you can clearly see that the stock Camera app (left) writes the wrong EXIF data, but the cropped image (right) has the correct data.