I noticed a tutorial of this for vray, and thought surely this can be done in mental ray....well it can. Often times I create a rendering with transparent objects, and want to use an alpha channel to add a background while keeping the transparency in the foreground objects. I'll show you how to do this with mental ray.
Now this is the image straight out of the render buffer, that I want to add an image into its background. The reflection is from an image that I plugged into an environment switcher shader. That way object inherit the images reflections, and the grey background that you see is just the environment background.
Max creates an alpha channel when a rendering is created. However, only certain file formats keep the alpha channel information (.png, .tga). I would recommend targa formats because it keeps the background, and has the alpha channel added in the same file...you can also save it as a 32 bit image which is a plus.
This is the background image that I added behind the snow globe. This can be done in any software. For this example I used Photoshop.
Now this is the image out of Photoshop with the snow globe and the background image using the default alpha channel. This is the problem we get with transparent objects such as glass...notice how the background is the grey from the origianl rendering and not the lights from the background image.
The trick to getting transparency in mental ray is found in the A&D shader of the transparent objects. Scroll down to the Advanced section and check Transparency propogates Alpha channel. That's it!
Now when you render, and check the alpha channel you will see that it is transparent for the materials with the new setting checked. Note: this doesn't work if materials have a map in the environment slot.
Now when you use the new alpha channel for your render, it will maintain transparency for your foreground to view the background image!
Here is the max file: 04-transAlpha_max2010.zip