Sunday, 1 November 2015

Modifying the default enum attribute values in Maya

So, when you add an enum attribute to an object in Maya you get a default pair of values "Green" and "Blue". I like to use enum attributes for switches on rigs, eg visibility toggles etc. It works great and you get the cool little popup menu. You can add as many options as you wish as well so it's nice and flexible like that.

But... Wouldn't it be nice to have the default values something more useable? Well, you can. So here's how I tracked down where they come from, so they can be changed. (This is my usual method for finding out how something core to Maya works)

Make Maya give us more info

First, go to your script editor window and turn on echo all commands. Also clear the history at this point so it's easier to see what it writes next.

This will mean we get more feedback from Maya when we click on stuff! So we can use that info to find out what scripts/ commands are being used.

Add an Enum attribute

Create a random object and add a new attribute

Check the script editor

Go back to the script editor and look at the output window. Opening that add attribute window has given us some info;

We can see Maya has run a couple of commands. AddAttribute and dynAddAttrWin look interesting. Now, we know that the add attribute window knows about the default enum values Maya has to offer (it writes them in a text box). So lets find out where that dynAddAttrWin command lives and what it can tell us.

The whatIs command is our friend!

The whatIs mel command is super useful. It will tell us the script filename of any command (as long as it comes from a file). In mel run whatIs dynAddAttrWi;

Look at that, now we can have a look inside the script that builds the add attribute window!

Find text that is in the window to find your UI

Right, looking at a bunch of UI script isn't very nice. But luckily there's an easier way to track down most elements. The answer is in the text we can read from the UI itself.

Check out the bottom of the window. There's our pesky default values. But also there's "Enum Names" written there as well. Most core maya .mel scripts also have a corresponding .res.mel file. This is where all the ui text information is stored (so it can easily be changed into another language). They always have the following naming convention, so they are really easy to find:

main_file_name.res.mel - simple!

If you search this file for the "Enum Names" text you will find an attribute that will be used in the main .mel file.

Now go back to the dynAddAttrWin.mel file and search for m_dynAddAttrWin.kEnumNames to see what we can find. It will be after a function called uiRes.

Getting closer, now without having to actually know how the ui works we've found the layout for the enums!

This section looks interesting, it seems to be getting some basic info about enums and then passing it to ui elements.

As far as UI elements go, that list of enums is probably a textScrollList, and then in the script above theres some reference to a ScrollList, but to be sure we're heading in the right direction lets click on it and see what echo all commands has to say about it.

Ok so we got a big long name there, also mentioning a textScrollList. Now we can use the string variable from the .mel file as it's global (a global variable can be queried from anywhere), and compare what that says it is to the output windows result:

Yay they are the same thing, so we are definitely looking at the right bit of mel script! You may get different names, as they tend to change each time the window is made.

Nearly there!

Ok so now we've hit another mel command. So we whatIs that guy and see where it takes us:

Cool this looks like that place. If you look around in this file for the attrEnumScrollList command itself you'll see a few lines down a variable being set and it mentions default AND enums, all in the same line!

You'll find that command in the same file, and when you do you'll see it mentions our old pal the uiRes function.

Ooo look green and blue!

Hooray! we are finally there!  

You can see there's that default green and blue value. Now you can change those to whatever strings you like! If in doubt, save a backup of this file somewhere. Alternatively you can modify the main attrEnumScrollList_addModeDefault() function to return whatever you want!

This is basically the method I use for all my messing about with Maya. Searching through the ui (.res.mel) files is great as you can get to the files you need very easily without know exactly how things are working. I use notepad++ and just search for the text I see in the ui in all files in the maya directory. Sure it takes a while sometimes but it lets you mess around with things, which is cool.


No comments:

Post a Comment