Monday, October 5, 2009

The butterfly effect

Autodesk has been trumpeting Maya's Python API since 8.5. It's improved, but it's still buggy compared to the MEL API. This week we discovered a truly insidious bug.

userSetup.mel is a standard entry point for many studios. On startup, Maya looks in MAYA_SCRIPT_PATH and executes the first instance of this script it finds. We use this to initialize various proprietary plugins and tools. Starting with 2009, we've switched to using Python to develop the majority of our tools. So, it makes sense that Maya would also have a Python entry point similar to userSetup.mel. It does. Maya scans PYTHONPATH for userSetup.py and loads the first instance it finds.

Wait, PYTHONPATH? Why not MAYA_PYTHON_PATH? We don't want to necessarily expose modules compiled for Maya in other python interpreters. So, we decided to use userSetup.mel to wedge a set of paths from MAYA_PYTHON_PATH into PYTHONPATH and invoke the first userSetup.py we find from there.

This worked fine for nearly a year. Suddenly, last week, scenes began failing to open with errors about not finding required plugins. We traced the problem down to Maya somehow saving scenes without the "required" lines that indicate which plugins the scene needs. WTF?

After a few days of trial-and-error, we traced it further to a single line of code that invoked a method from the Python OpenMaya API. Without producing any errors or warnings, a standard API method call caused Maya to produce invalid output on save.