What you are doing is actually necessary. The image that is in memory is just a bunch of pixels and some metadata. If you want to read the information in another MagickFormat you first need to Write it to 'something'. I have no experience with mvc 5 but are you not allowed to write directly to the Response.OutputStream?
using (MagickImage image = CreateMagickImage())
{
image.Write(HttpContext.Current.Response.OutputStream);
}