I am not sure why IM is not detecting your stream as a jpeg. You can help IM by specifying the format before you load your image. You can do that with the MagickReadSettings class:
MagickReadSettings settings= new MagickReadSettings(); settings.Format = MagickFormat.Pjpeg; using (MagickImage image = new MagickImage(stream, settings)) { }