Although your observations are right Benjamin, don´t remove anything at the moment. In the last picture set the shirt looks very twisted. Most the back, wich seems to be too tight at the left and too wide on the right. But you also can see the center back ( the ironed fold) is not falling straight. Never noticed this in the other pics, so I think you just wear it twisted. Your shoulders arn´t stable as well. Somtimes your right sholder is lower, sometimes your left, so it doesn´t make sense to do a dropped shoulder alteration.
To sort out wether it is fit or the wearing, please fold your left front exactly at the center front line ( the line where the buttonholes are positioned) and baste it. Also mark the center front on the other front piece. When fitting, pin the fold exactly on the marking, and make it hang strictly vertical in the center of your body. also make sure the center back is running straight and vertical and the back neck point is at the right place.
As Greger wrote let the shoulder seams end 1/4 inch before the neck line, thus leaving the SA unsewn.
A profile pic of both sides is also useful to recognize irregularities, as is a marked chest line on the shirt.