@robdewinter1 (Apple’s SHARP ai model can turn 2D photos in 3D Gaussian Splats models) Video by Rob de Winter
My Notes: Apple’s SHARP ai model can turn 2D photos in 3D Gaussian Splats models
This is absolutely incredible. @apple ‘s new SHARP model (open source, single image to 3D) lets you generate 3D Gaussian Splats with full depth information in just seconds, running entirely locally on your own machine (!).
I built a custom interface where you can drag and drop images, plus a Three.js viewer to navigate through the splat in real-time. Built with a little help from Anthropic’s @claudeai . You can even open it in fullscreen for a closer look, and adjust the splat/point size.
For fun, I tested it on iconic paintings by Dutch masters: Escher, Vermeer, and Van Gogh. The results were stunning, I think (especially when you realise it only took seconds to generate locally on an M1 Max MacBook Pro and it even analysed this magical Escher drawing). I also tried photos from one of my favorite shows of the year, Severance (at the end of the video).
I think this technology will be invaluable for VR experiences, but also as reference material for AI image generation, and probably a lot more I haven’t even thought of yet. This feels like a small piece of something much bigger.
What would you use this for?
Instagram by: Rob de Winter (@robdewinter1)
Source: https://www.instagram.com/reel/DSr_8XxiA7T/?igsh=anVjdnh3eTUxYTA3