Adding Basic Japanese Visemes to Avatars for VRChat
A lot of avatar bases are sold primarily to English speaking audiences, and may not support Japanese visemes for natural things like speech using the JA locale, or worlds that rely on these blendshapes to animate your character for you (such as MMD worlds.)
Getting Started: What You Will Need
- A fully rigged 3D model or premade avatar base ready for VRChat
- Blender 2.93
- CATS plugin for Blender
- PumkinsAvatarTools for Unity
- The proper version of Unity for your avatar/VRChat
- Roughly 30 minutes
Why Blender 2.93?
Blender version 2.93 is a LTS release of Blender that still uses Python 3.9 which does not contain some regression bugs that break CATS's FBX exporting tools. If you downloaded Blender via Steam, you can right click on it in your Game Library and select Properties and then Betas. From here you can specify which release to use (2.93 - Stable - LTS)
If you have a different version that you know works with CATS's FBX exporting tooling, feel free to use that instead. I'm not a cop.
Installing the Plugins
CATS
Open Blender and navigate to the Edit menu, and then click Preferences. Once your preferences window opens, navigate to Add-ons and click Install. Point Blender to your downloaded .zip file. Once installed, ensure that CATS is enabled. It should pop up on the right hand menu of the default scene view (with the cube.) If it doesn't appear, look for an arrow to click to expand it.
PumkinsAvatarTools
Download the .unitypackage and import it into your project. You're done! If you have an older copy from before PhysBones, you'll probably want to update it.
Editing the Model
Once you've got your environment set up, you're ready to begin. Import your model's mesh (probably in the form of an .fbx file) into Blender. If it's an FBX, it will look something like File > Import > FBX. You can delete the default cube, light, and camera.
You'll need to navigate through the Armature (located at the top right of the default view), and into the Body in order to expose the Object Data Properties menu (the green upside-down triangle.) Once you're there, expand the Shape Keys.
All of your Shape Keys should have their value set to 0 before proceeding. If you're familiar with Blend Shapes, we're duplicating the standard vowel Blend Shapes into new ones that the game can recognize. Select the Shape Key you wish to duplicate and set the value to 1 so that it's "on". Once you've done that, click the downward arrow below the minus button. This will open a menu where you can select New Shape from Mix. Refer to the following table for a list of what to duplicate, and what to name them. Remember! Once you make a new Shape Key, ensure you set the values back to 0!
| Shape Key to Duplicate | New Name |
|---|---|
| vrc.v_aa | あ |
| vrc.v_ih | え |
| vrc.v_oh | お |
| vrc.v_ou | う |
| vrc.v_nn | ん |
| vrc.v_ee (optional) | い |
You may have noticed that this is not nearly all of the visemes that are available. We're being lazy in this because the Japanese language mostly follows the Consonant, Vowel pattern for its letters. Why put in ra ri re ro rou when a i e o ou works the same? I'm sure there's a good reason to, I just don't know enough to tell you with confidence in a random blog post.
Exporting
CATS makes exporting easy. Just expand the CATS menu, click "Export Model" (if it complains about material slots, just click "Continue to Export"), and save it somewhere. That's it! You're done!
Unity – Applying Your New Mesh
If you're feeling cozy with your model, you can just drag in the new Mesh to your project and set it up as normal.
For those of us who aren't super comfy, there's PumkinsAvatarTools.
The first step is to show the PumkinsAvatarTools menu which can be done by clicking Pumkin up at the top bar, Tools, and then Avatar Tools. Go ahead and dock this where it's comfy for you. Drag in your new mesh into the world, and then into where it says "Avatar" in Pumkin's Tools. Expand the "Copy Components" accordion, and drag your working avatar (the one that's using your old mesh, the one without the new visemes) into the Copy From field.
It's up to your discretion to make sure that everything transfers over properly. I've personally had problems where it's added a duplicate FX layer to the avatar, but it's a simple fix. Make sure that you hit "Select All" (or whatever's appropriate) and that if you have complex PhysBones interactions that you ensure that that transfers properly. Once your avatar is back up and running and ready to go with your new mesh, congratulations! You're all set to test in an MMD world. If It worked, you're all set :)