With Houdini 20 we got our first official machine learning node from SideFX. While we previously had example scene files that did machine learing related stuff or plugins like mlops, this is the first machine learning tool that you get just by installing Houdini. So we should take a look at what it does and what we can use it for!
Links mentioned in the video:
MNIST and Style Transfer
MIDAS Depth Estimation
So how did you guys do the mushroom in the title picture of the class? Is that using ONNX?
Hey! You can take a look at the scene file for a breakdown of this. The mushroom model is a free 3d scan from aixterior. The “hologram points” on the right are created by projecting a 224×244 grid of points onto that scan, grabbing color info from it’s texture and running that through the style transfer model.
Which style transfer model are you using there?
It’s still one of the ready-to-use models from the repo. I think either “Udnie” or “Rain Princess”. I’d simply choose the one that works best for the picture I’m loading in. If you dig a bit deeper in this repo, you’ll also find fairly good instructions on how to train your own models and export them as ONNX files.
are there any onnx of higher resolution like at least 1000×1000 for image manipulation
Hey Matia, the resolution in case of the style transfer model is set by the onnx export, not the model itself. I haven’t found higher resolution exports online yet (though I haven’t spent a lot of time looking), but if you’re somewhat comfortable with Python, you can export your own onnx version of that model with any resultion. Take a look at the guides (specifically under “Training”) here:
https://github.com/onnx/models/tree/main/validated/vision/style_transfer/fast_neural_style