New Post: Fonts not being used when using Pango

I'm using pango to generate some text on a server. When I run it from my local machine, or in the context of my user name on the server, everything runs fine.

However when the image is generated on an IIS site, or from a task run under the context of a system account with less privlages. It falls back to a different font. It seems to be an issue of rights, but I don't even know where to begin to check.