Pure Data patches in Unreal Engine 4 via Wwise
EDIT MAY 2022: A lot of time has passed since this post, and an update is needed. Heavy is no longer hosted online (although you could get it from github and compile it locally) and Unreal Engine 5.0 has been released to the public. If you are looking to do some procedural audio or dynamic music in Unreal, I highly recommend checking out Metasound and Quartz (looking for a quartz tutorial? click here). These system are not only shipped with the engine and ready to use, but are going to be supported and improved in future versions of the engine, hence I highly recommend them.
Pure Data represents a valid and accessible solution to create procedural audio systems. Running it in a game engine like Unreal however could be quite confusing, especially if you are not familiar with programming. In this tutorial, I’ll show you how to convert your PD patch into a plug-in for Wwise, and how to make it run into Unreal. This guide is meant for the Windows versions of the softwares.
Before we start
This method was tested with UE 4.13 and Wwise 2016.2.1.5995. I didn’t have the time to try it with different combinations, but it should work for any version of Wwise integrated in UE using the automatic tool present in the Wwise launcher.
The guide is structured so that anyone with a basic knowledge of Wwise and Unreal can understand it. You won't need to know how to code to follow it. The method consists in converting your PD patches into Wwise plugins via the Heavy compiler, and then compile the Unreal integration of Wwise to recognize them in the engine. Here's a brief index so anyone can skip directly to what they need.
0. Requirements: what you need before to start
1. Upload your patch to Heavy: quick illustration of how Heavy works and how to prepare your patch for it. Includes a really nifty laser gun PD patch to download, used for examples.
2. Your patch in Wwise: shows how the patch is added to Wwise and how the parameters are handled. Illustrates the problem of the absence of bangs and a couple of ways to deal with it.
3. Your PD patch in Unreal: shows what you need to do to have your patch working in Unreal.
4. Laser Gun patch in UE4: quick implementation of the laser gun patch in UE4.
5. Conclusions.
0. Requirements
This is what you will need to convert your patch.
1 A C++ Unreal Project: to use custom plug-ins in the Wwise integration for UE4, the project’s code needs to be rebuilt from a compiler. If you don’t feel confident with code, don’t worry: what you need to do is pretty simple and will be explained later. If your project is a Blueprint only type, you can convert it by following this guide.
2 A version of Visual Studio: the guide linked above suggest visual studio express 2013. Visual studio community 2015 and 2017 are ok as well. Just be sure to check the C++ option when you run the installer.
3 A Wwise project linked to your UE4 project. The process is pretty straightforward and can be made directly in the Wwise Launcher.
1. Upload your patch to Heavy
What you need to do is to register to the Heavy website and upload your patch. Before you do so, however, here is a quick explanation on how to prepare your patch.
EDIT: Since I wrote this guide, the Enzien Audio website and online compiler have been taken down, so they are no longer available. The wonderful people at Rebel Audio, however, are hosting an Online Heavy compiler, you can therefore convert your patches by following this link .
When the patch is converted, your regular dac~ node will be routing the audio out. However, you want to expose some parameters in order to control them from Wwise and to trigger your bangs. To do so, you need to use dedicated receives annotated with the @hv_param command. For example, if you want a gain control to use in Wwise, you will have to create a node like this.
Thanks to the receive node, in Wwise you will find a gain parameter going from 0 to 1, with a default value of 0.2. That parameter can then be associated with RTPCs and all that cool stuff.
In general, the syntax to use to expose parameters is the following:
r nameofparameter @hv_param minvalue maxvalue defaultvalue
You can see below a really poor sounding ghetto-made laser gun (download): it simply generates white noise and runs a sweeping filter to it. The user can specify the starting frequency of the sweep, its resonance, and a random amount that will be added to the base frequency each time the patch is triggered. Take a look on how I exposed the different parameters, and look to the trigger one. The problem with heavy and Wwise right now is that you can send only numbers and not bangs. I found two ways to overcome it. The easiest one is just to link a @hv_param receive directly to a bang. In this way, anytime you will change the value of that parameter in Wwise (or in UE4) you will trigger a bang. Sometimes, you might want to use a spigot 0 node like I did. I’ll come back on why later.
When your patch is ready, upload it to Heavy and download the “Wwise Windows 64 bit” zip folder.
2. Your patch in Wwise
In the zip Folder, you will find different files. Copy the one named like this:
HV_{PATCH_NAME}_WwiseSourceAuthPlugin.dll
HV_{PATCH_NAME}_WwiseSourceAuthPlugin.xml
In the folder:
C:\ProgramFiles(x86)\Audiokinetic\Wwise\2016.1.0.5775\Authoring\x64\Release\bin\plugins
Open your Wwise project now, and add a new sound sfx where you like. You will find your patch in the content editor under add source. Add it, and you will be able to see all your exposed parameter. Sometimes, if they are more than three, you will need to double click the row to see them all (you can see in the picture that my row has only three parameters, but in the popup window I have all of them).
From here, you can assign all the parameters to your RTPCs, like any Wwise plugin. Press play on your sfx and you will hear your patch. This is why I used spigot: you might not want to trigger your bangs when you play your event. If the number going into the spigot is 0 by default, you won’t hear anything, it will only play when changed.
The way the laser gun works now is by using play to activate the patch, and then by moving the trigger number to (surprisingly) trigger the sound. You might wonder why I haven’t just avoided spigot and had the patch play anytime I use a play event. I noticed that in this way I was losing the random function, and every time I pressed play the sound had the same frequency! In this way, instead, it works correctly.
Check if your patch works like you want in Wwise: if it does, you are now ready to run it in Unreal!
3. Your PD patch in Unreal
In this section you will need to write some code and to build it in order for your patch to run. Don’t worry though: it’s easy as dragging some files around and writing on your keyboard. If you feel not confident, remember to work on a copy of your project, to avoid data loss. Also, use Visual Studio community 2015: it's the version I am using, so it should be the easiest to follow. Remember: your UE4 project must be a C++ one and not a blueprint only.
Take the zip folder downloaded before. You will need to copy some files in the Wwise plugin folder inside your UE4 project folder.
Copy the .h file in
D:\...\Unreal Projects\YourProject\Plugins\Wwise\ThirdParty\include\AK\Plugin
And copy the .lib file in these different folders
D:\...\Unreal Projects\YourProject\Plugins\Wwise\ThirdParty\x64_vc140\Profile\lib
D:\...\Unreal Projects\YourProject \Plugins\Wwise\ThirdParty\x64_vc140\Profile(StaticCRT)\lib
D:\...\Unreal Projects\YourProject \Plugins\Wwise\ThirdParty\x64_vc140\Release\lib
D:\...\Unreal Projects\YourProject \Plugins\Wwise\ThirdParty\x64_vc140\Release(StaticCRT)\lib
Ok, we now need to tell to the Wwise plugin for UE4 that our patch is there!
First, we need to specify where the .h file is. With Visual Studio, open the file:
D:\...\Unreal Projects\YourProject \Plugins\Wwise\Source\AkAudio\Private\AkAudioDevice.cpp
Now look for the line saying:
// Add additional plug-ins here.
When you find it, type this line below it:
#include <AK/Plugin/nameofyourHfile.h>
Which in my case is:
#include <AK/Plugin/Hv_lasergun_WwiseSourcePluginFactory.h>
And save.
We need know to tell the plugin to look for the .lib file. To do this go to
D:\...\Unreal Projects\YourProject\Plugins\Wwise\Source\AkAudio\AkAudio.Build.cs
Around line 230, you should find a list of commands that looks like this:
AddWwiseLib(Target, "nameofplugin");
Add another one at the end, and change the nameofplugin (keeping the quote marks) with the name of your lib file without the .lib extension
In my case it is:
AddWwiseLib(Target, "Hv_lasergun_WwiseSourceEngine");
And for this one don’t forget about the semi column after! (I always do)
We are almost there! You now need to rebuild your UE4 project from Visual studio. In the project folder, you should find a .sln file (if you don’t,be sure your project is a C++ one, or right click on your .uproject file and hit “generate visual studio project files”). Open it with Visual Studio. Hit "Build>name of your project". If all goes right, you should see something like this in the output window.
========== Build: 1 succeeded, 0 failed, 0 up-to-date, 0 skipped ==========
4. Laser gun patch in UE4
You can now use your patch in Unreal as you would do with any other Wwise source plugin! Welcome to the wonders of procedural audio!
Create an event to play and activate the sound sfx object containing your plug-in (aka the PD patch) and then use RTPCs to modify the parameters.
Here’s a quick example of how I implemented the laser gun.
On begin play an event is posted to play the patch. As mentioned before, this is needed just to “set” it, it won’t play any sound thanks to the spigot.
A triggger RTPC is linked to the trigger parameter of the patch: when the right click is pressed it is set to 0 and will play the sound, moving back to 1 when released.
5. Conclusion
This should get you started with PD in UE4. The same method works also to include third party Wwise plugins: you usually have already all the files there, it just needs the coding bit!
There might be different or better ways to do this, so feel free to contact me if you find one, if you have any problems or want to give me some feedback!
