So for the last 2 months, I have been working with voice actors to record all conversation in the game.

The main reason why I did this was to adds another level of personality to each character and If I am being honest reading dialogue without voice acting sucks.

When you have a game filled with lots of reading the player has to already be invested in the story or they will start skipping a lot of it text. I Found this was one of the major pitfalls in VN and it’s was even worse for VN with any sort of porn reward (yes they have stories too). Visual novels aren’t the only ones with this issue, I was playing “pillars of eternity” the other day and it just killed me with the amount of pointless writing. Sure, the writing good but If I am bored reading your game, it really doesn’t matter how good the writing is because at this point I am skipping it to get to the good parts.

I built my game so that the dialogue wouldn’t rambling on pointlessly and it was long enough to deliver a message or a feel within every script. Also by adding choice through the dialogue not only breaks up the reading, it also helps re-engage the player again with the story.

That alone is not enough, you need to add lip syncing and facial animation to really help the storytelling. Huniepop did a great job of making the character come alive and my goal is to one day make an old Bioware style game like Kotor or Dragonage 1. So having the character’s face animate while talking is super important to me. The sad truth is Bioware is never going to make one of these RPG’s types of games again. Their current type is less about RPG / storytelling and more about Action (barely RPG) open world which sucks because I like their old stuff more than their new; there is going to be a demand for these old type of game in the future.

So here is how I pulled off face animation and lip syncing for my game. Hopefully, by me share how I did this from scratch some developer that wish to make a game with better story element will find this and learn from it.

---

Here is how to make facial animation and lip syncing in your game using Melanie level up 1 script.

---

1. Get a program call Papagayo. I think the pros (Bioware) uses FaceFX and Huniepop has a one from the unity asset store ( might be wrong but there is one on available in the store). Papagayo is free.

2. Drop the audio and text into the editor. Make sure you set the frame rate to 60. (assuming your game update loop is every 60 seconds)
side note: trim the fat (sides, long pauses) from the audio file. it just makes your game bigger for no reason.

3. Make sure the last word is on the very end of the timeline if not you going to need to stretch to the green bar to fit which will reset all words on the timeline; make sure you do this first. Now go through each word and stretch, shrink and move them to the correct location. This will make the voice actor sound like they have speech impediments as they say the word “the” 10 times before it sounds right.

4. Once you have it sounding right and looking right, export it.

5. Cleanup the file by removing the “mohoswitch1” make sure the first line index is 0 and that double rest are removed so that everything should be in numerical order.

Side note: The MBP, E, L, AI, Etc, E, you see beside the number are the lip shapes that need to be made on that tick.

Before:

After:

6. Time to add animation. I create a tool to make this easier for me (take the day to create a tool… editing the exported file by hand sucks). Under the hooded what it does is it create a list in game that keeps track of the mouth index and the mouth shapes (export file). Then on the update, it moves a counter and checks to see if it match the next mouth index. If it does it changes the mouth. This principle can be used for eyes and eyebrows as well as special effect. To make life easier I create a timeline with each mouth position place on it as a red square and a way of exporting the file again with the correct face animation added.

In the end, you should have something that looks like this.

Now that you have the steps go out there and make a better game with lip syncing and animation!