Hugues Bruyère
@smallfly
Followers
7K
Following
5K
Media
1K
Statuses
5K
Try / Share / Iterate — Creative Technologist / Chief of Innovation @bonjourdpt
Joined February 2009
Mixed Reality + Diffusion prototype as a tool for exploring concepts, styles, and moods by transforming real-world surroundings into alternate realities. #MixedReality #MR #AI #StableDiffusion #Quest3
18
58
441
'3D Gaussian Splatting for Real-Time RF Rendering' rose to the 2nd challenge I presented. The magnifying glass isn't still crystal clear, but the result is compelling. Look at the small details/hairs on the succulent plant. Rendered in real-time. #gaussiansplatting #3D #AI
40
304
2K
Finally accessing Mirror Teleportation and Dimensional Storage superpowers. Look at how the reflection in the mirror has been reconstructed and synthesized - I'm able to get in the reflection! NeRF / Instant-ngp getting out of control...#AI #NeRF #synthesis #volumetric
64
393
2K
Simple repulsion force revealing the colors within the 3D Gaussian splats. Real-time rendering in Unity. #3DGaussianSplats #madewithunity #unity3d #3DScan #memories #Montreal
23
160
1K
With great power comes great responsibility .— Goofing around with the new @Depthkit pro and Kinect Azur.— Realtime handheld AR .— @bonjourdpt . #augmentedreality #mixedreality #AR #XR #depthkit #kinect #arkit #madewithunity
10
248
1K
Transparency and wireframes.— Realtime handheld AR . @bonjourdpt #spatialcomputing #mixedreality #augmentedreality #3dscanning #realitycapture #madewithunity
7
148
716
@muratpak @archillect Octreefied XVI ft. Ben⠀.More volumetric capture and realtime render experiments here on Twitter or IG -
1
113
664
Time warp — A multi scale AR exploration in which the user position affect the flow of time. — When moving away the user is going back in time. — 7th iteration. #AR #AugmentedReality #MR #XR #ARKit #ARCore #photogrammetry #3dscan #MadeWithUnity #Unity3D #Montreal @bonjourdpt
7
111
517
New dataset w/ a magnifying glass. The outcome isn't as refined as the last one. The magnifying fx extends beyond the actual FOV of the glass, yet it is magical. Rendered in real-time w/ '3D Gaussian Splatting for Real-Time Radiance Field Rendering'. #gaussiansplatting #3D #AI
10
57
515
I'm still amazed that I can spontaneously capture a location or moment, in just 2min, and then reconstruct the scene and freely navigate through it.. NeRF trained and rendered w/ @nerfstudioteam + nerfacto model.. #AI #NeRF #nerfstudio #neuralrendering #montreal
14
57
438
This time, I took the prototype to the backyard—opening portals, transforming my surroundings into a series of alternate realities. #MixedReality #MR #AI #StableDiffusion #Quest3
8
48
439
Sliced Arnaud — 2nd iteration.— AR exploration inspired by Julian Voss-Andreae work.#ar #mixedreality #arkit #arcore #madewithunity #montreal
5
88
405
Part 1. Today, my son and I had a great impromptu session of hands-on hallucination. Back at it, exploring physical interfaces to interact with a real-time img2img diffusion pipeline. #AI #StableDiffusion #Realtime
6
45
397
Here's a simpler dataset of the Head of Zeus. Captured w/ a phone while visiting the ROM. The details become more impressive as you get closer to the sculpture. Trained & rendered in real-time w/ '3D Gaussian Splatting for Real-Time RF Rendering'. #gaussiansplatting #3D #AI #ML
12
33
370
In the past few weeks, I deep dived into an exploration revolving around the use of physical interfaces to feed and interact with a real-time img2img diffusion pipeline using Stream Diffusion and SDXL Turbo. What really captivated me is to use my hands, objects, art supplies,
8
74
369
Looking through the magnifying glass - New NeRF experimentation using @NVIDIAAIDev 's InstantNeRF. The result is far from perfect but still really interesting, at least for me. #AI #NeRF #InstantNeRF #neuralrendering #volumetric
9
38
350
Second test using the @nerfstudioteam and Blender pipeline. Same scene as the previous video but now using a handheld virtual camera - nearly infinite POV options!. I've also resolved the issues I had in the 3D scene. cc @bonjourdpt. #AI #NeRF #nerfacto #neuralrendering #b3d
7
35
337
I finally managed to run some of my own datasets with '3D Gaussian Splatting for Real-Time Radiance Field Rendering'. Here is the first one, rendered in real-time. The output is incredible! More to come. #AI #NeRF #gaussiansplatting #radiancefield #Computervision
6
34
333
My first MR experiment outdoors, in an alleyway. and my first time wearing a VR headset in the street. Though it still feels awkward, I find the experience far superior to AR. The user's position controls time.. #MixedReality #AR #XR #MadeWithUnity #3DScanning #MetaQuest3
8
48
324
In this iteration, I’ve added @AIatMeta’s SAM 2 to our workflow to segment people. Lisa and I had fun testing it in our lab—striking poses in sequence, with our bodies precisely isolated for a refined Stable Diffusion effect. #MR #AI #Passthrough @MetaforDevs
13
32
334
I swear, last one for today. But look at the details and sharpness of this NeRF. @NVIDIAAIDev Instant-ngp and DLSS bring the process of capture to real-time rendering to the next level. ~25min from the 1st photo to this. #AI #NeRF #neuralrendering #InstantNeRF #dlss #realtime
8
45
317
Nick into the multiverse — Training a NeRF model and messing with it by using photos that include a mirror. Using @NVIDIAAIDev's Instant NeRF. #AI #NeRF #multiverse #synthesis #volumetric cc @nsroy @bonjourdpt
14
37
311
The 1st prototype of the Instant NeRF Photomaton is up and running at @bonjourdpt. Here are some portraits manipulated in real time - ~2min after the capture. NeRF trained using @NVIDIAAIDev Instant-ngp.. #photomaton #NeRF #portrait #selfportrait #selfie #AI #InstantNeRF
12
61
306
Physical light + real-time diffusion exploration - A thread. (1/n). #StableDiffusion #SDXLTurbo #StreamDiffusion #Realtime #nivdia
6
46
292
Three years ago, I captured Montreal's 'Radio Doctor' shop—one of those special places that make my neighborhood. Since 1960, Joseph Hovsepian has been repairing radios, turntables, and more. Here's the shop reconstructed as a 3D Gaussian splat. #gaussiansplatting #3dgs
7
28
295
Wrapping up this weekend's dive into '3D Gaussian Splatting for Real-Time Radiance Field Rendering' with this echinacea flower dataset, captured in the void. Look how sharp the petal edges and the cone are!. Link to the repo in the following post. #gaussiansplatting #3D #AI #ML
9
22
288
Today's iteration on refining my 3D Gaussian splats pipeline. Same dataset but this time, I’m pushing the level of detail while focusing on a smaller section of the scene. I can now get really close. Rendered in real-time using PostShot.
2
26
285
Today's impromptu capture and NeRF. I can't stop! Trying to capture and synthesis the exterior and interior of a subject/scene - the car and behind the glass of the garage. Using @nvidiaai 's Instant ngp...#AI #NeRF #nerfies #neuralnetworks #volumetric @bonjourdpt
7
48
275
It’s raining summer. Winter is coming, but I’ve captured a bit of summer. Rendering all this data in real-time with Unity, I had to start playing with it. #3DGaussianSplats #madewithunity #unity3d #3DScan #memories #Montreal
6
40
278
Playing with depth and layering—using the Mixed Reality + Diffusion prototype to explore concepts, styles, and moods. #MixedReality #MR #AI #StableDiffusion #Quest3
12
42
275
New trained model for the NeRF portrait series - this one ft. @zacharylr. Check the specularity of the jacket as the camera rotates around. #NeRF #neural #synthesis #volumetric
7
13
254
Tonight I grabbed some other ingredients: milk from the kitchen and black ink from my son. Watching the images emerge is like clearing my mind. Some parts of this video have been sped up 4x, while others are in real-time. Real-time diffusion with Stream Diffusion + SDXL Turbo.
7
41
260
Here is a night dataset, captured at the botanical garden, during the 'Jardins De Lumière' event. 152 photos shoot handheld using the Samyang 12mm f2.0. 'Trained' & real-time rendered w/ '3D Gaussian Splatting for Real-Time Radiance Field Rendering'. #gaussiansplatting #AI #ML
9
27
251
Finally tried a large scene with '3D Gaussian Splatting for Real-Time Radiance Field Rendering' — Barbentane's windmill, built in 1774. It's looking sharp. The details in the distance are impressive. #gaussiansplatting #radiancefield #AI #ML
3
29
242
Early test run with freshly added depth estimation — Mixed Reality + Diffusion prototype to explore concepts, styles, and moods. #MixedReality #MR #AI #StableDiffusion #Quest3
11
32
252
Hands-on Hallucination session: New camera angle, magic clay, and an additional light source. #AI #StableDiffusion #Realtime
8
25
260
Ran '3D Gaussian Splatting for Real-Time Radiance Field Rendering' using a dataset from our proof-of-concept rig @bonjourdpt. Just 13 HD imageswere used! Check out the dog's details, but @nsroy, you're still the best-looking subject I've worked with!. #gaussiansplatting #AI #ML
3
26
243
Continuing my 3D Gaussian splats pipeline iteration. This time I get closer. Here’s a quick test comparing training steps: 12 million splats + 60k steps vs. 6 million splats at 40k steps. The number of splats clearly impacts the level of detail. But I keep in mind the
6
35
245
In the past week, I've been focused on iterating on the necessary pipeline to train 3D Gaussian Splats with the best level of detail and the least amount of noise (the so-called 'floaters'). This touches elements at every step of the pipeline: capture, dataset pre-processing,
4
24
230
This is getting interesting. In this iteration, the magnifying glass is tracked in real-time, feeding only what is seen through it to the diffusion process, revealing the invisible within a piece of clay. Real-time diffusion with Stream Diffusion + SDXL Turbo. #StableDiffusion
8
31
222
Exploring the use of physical interfaces, depth, diffusion, and mixed reality. #MixedReality #MR #AI #StableDiffusion
4
27
219
NeRF + magnifying glass experiment. In that one, you can see the magnified painting fading in, over & behind the original one. At the end, the depth in the painting feels real - the trees pop out. Trained w/ @NVIDIAAIDev's InstantNeRF. #AI #NeRF #InstantNeRF #neuralrendering
6
36
208
𝗗𝗼𝗿𝗺𝗮𝗻𝘁 𝗠𝗲𝗺𝗼𝗿𝘆 𝗩𝗶𝗲𝘄𝗲𝗿: Because I almost always have some gear with me—a drone, a mirrorless camera, or at least a phone—I can capture scenes on the spur of the moment. Here I am freely flying through 3D Gaussian Splats scenes trained from these impromptu
3
22
199
First test at mixing 3D gaussian splats and geometry. Behind the scene — real-time rendering of a radiance field in Unity using 3D gaussian splatting. Thanks @aras_p for the Unity implementation. #gaussiansplatting #radiancefield #Unity
4
32
190
The last 6 months have been so much fun experimenting with @NVIDIAAIDev 's InstantNeRF. Here are short clips of most of the NeRFs I have trained, packed in a less than 2 min video - no specific order. cc @bonjourdpt . #AI #NeRF #InstantNeRF #neuralrendering #volumetric
7
29
185
Neural rendering of a night scene, using 'Instant-NGP'. I found the light amazing. I like the clear delimitation between the exterior (in focus) and the interior (out of focus). This one looks even more like a galaxy. .#NeRF #neural #synthesis #volumetric
6
27
185
Today's image synthesis - 20042238. Here are the Stable Diffusion and NeRF renders side by side. From a walk in the snow with a 360 camera to a NeRF, then a render with minimal VFX and finally a touch of Stable Diffusion hallucination. ..#NERF #StableDiffusion #VFX #AIart
6
22
183
NeRF of Barbentane's windmill, built in 1774. One of my best. — Rendered Memories VII.— Mix of mirrorless and drone stills.— Neural rendering using @NVIDIAAIDev 's InstantNeRF. #AI #NeRF #InstantNeRF #neuralrendering #memories #windmill #barbentane #render #volumetric
9
21
187
Behind-the-scenes of the NeRF I shared yesterday, from capture to render — from a 2min 360 video done using an @Insta360 RS 1'' to NeRF trained with @nerfstudioteam. This should answer some of the questions I received. cc @bonjourdpt. #AI #NeRF #nerfstudio #neuralrendering
5
25
181
The scanner scanned. 3D Gaussian Splatting, rendered in real-time within Unity, both in the scene view and the game view. Overlaid is a screen recording of the same scene being explored and rendered in real-time as a NeRF using Instant-ngp. #gaussiansplatting #NeRF #cv #ML #AI
2
20
185
Today my son took over the Mixed Reality + Diffusion prototype… First stop: Mario Kart. #MR #AI #Passthrough @StabilityAI @MetaHorizonDevs
4
19
176
Every century, on the same day, at the same time, at the very same place. . NeRF ➜ Stable Diffusion ➜ Temporal Kit + EbSynth. . #ai #NeRF #stablediffusion #TemporalKit #Montreal
6
40
167
Behind-the-scenes look at the screen recording I shared earlier.. Using magic clay to create layers of shapes and color as a base for revealing landscapes and hidden worlds. Only what is seen through the magnifying glass is fed to the img2img diffusion process.
4
26
158
NeRF of the "En Masse - Mural". Trained and rendered using NeRFStudio and nerfacto - @nerfstudioteam . .#streetart #graffiti #NeRF #nerfstudio #nerfacto #neuralrendering #montreal
3
14
142
Iterating on my 3D Gaussian splats pipeline using an iconic Habitat 67 dataset, shared by @RealityCapture_ for educational use. What's better than a local dataset?. This is just the first step in my current sprint. Alignment done in RealityCapture, with 3D Gaussian splats
2
19
153
Today's '3D Gaussian Splatting for Real-Time Radiance Field Rendering' test using a selfie dataset shot with a drone. Rough camera motion, but it's running in real-time. #gaussiansplatting #radiancefield #AI #ML
2
20
145
The Giant Fish nebula — Memory fragment showing @waxheadart in action, painting a giant fish. You can also see one of @cryote_pup characters. Novel view synthesis using @NVIDIAAIDev 's Instant NeRF. #AI #NeRF #nerfies #synthesis #neuralrendering #volumetric #portait #streetart
8
20
142
Octreefied XVI - Ben⠀.— Real-time point cloud rendering using Unity VFX Graph⠀.— @bonjourdpt⠀.⠀.#3dscan #photogrammetry #portrait #realitycapture #pointcloud #madewithunity @unity3d #realtime #experimental #creativecoding #digitalart #glitch #glitchart ⠀
4
22
134
Training 3D Gaussian splats of the alleyways in my neighborhood. It's captivating to see them emerge from the noise.
7
13
140
Quick '3D Gaussian Splatting for Real-Time Radiance Field Rendering' test using this snail shell dataset, captured in the void. Trying small subjects, before going big. #gaussiansplatting #3D #AI #ML
4
12
136
Stay home - There is no escape. ⠀.— Playing with the Oculus Quest pass through camera and guardian system⠀.— Room scale point cloud rendering⠀.— Social Distancing Day 52⠀.— @bonjourdpt⠀.⠀.#virtualreality #vr #xr #oculus #oculusquest #3dscan #madewithunity #realtime
5
21
131
Playing with transparency and wireframes - 2nd iteration.— Realtime handheld AR . @bonjourdpt #spatialcomputing #mixedreality #augmentedreality #3dscanning #realitycapture #madewithunity
5
26
128
First prototype using the Quest 3 passthrough feed — Captured snapshots are processed in real-time with a Stable Diffusion ComfyUI workflow on a local machine and sent back to the Quest. Video at 2x speed. Next: optimization, etc !. #MR #AI #StableDiffusion #ComfyUI #Quest3
7
15
140
Revisiting past self-captures & portraits, many from covid's peak. Rendered in real-time with '3D Gaussian Splatting for Real-Time Radiance Field Rendering'. Notice the motion blur in the backgrounds, while the scene's center is crisp. #gaussiansplatting #radiancefield #AI #ML
4
20
135
Just tried the new @nerfstudioteam & Blender pipeline - so much potential! Excited to explore it for some ongoing projects at the studio/@bonjourdpt. Some issues with what I did in Blender. I'll look into that for next time. #AI #NeRF #nerfstudio #neuralrendering #blender
3
11
132
Manhattan, 2023.11.02 — My hotel room, lost in the city. Real-time 3D Gaussian Splatting. Audio by my son. #gaussiansplatting #radiancefield #nyc
1
22
132
Freshly trained – this 3D Gaussian splat of a pine cone is packed with detail! I love how close I can get without losing quality.
2
9
131
Messing with the mighty Zeus! Diving into the shaders to achieve real-time deformation of 3D gaussian splats. Just scratching the surface here. #gaussiansplatting #radiancefield #Unity #VFX #3D
0
11
129
Breaking reality .— Mirrorworlds.— Realtime handheld AR . @bonjourdpt #spatialcomputing #augmentedreality #3dscanning #realitycapture #arkit #madewithunity
2
32
127
@muratpak @archillect The second half. Octreefied XV ft. Greg⠀.More volumetric capture and realtime render experiments here on Twitter or IG -
2
16
123
Time to update my Quest MR + Gen AI prototype now that @MetaQuestVR Camera API is out — This proto is using a workaround to capture passthrough snapshots, processes them with Stable Diffusion, and displays the results back in their exact physical location. #MR #AI
5
14
128
It's a Xmas miracle! Just finished setting up this handheld scan rig and it's a beauty. Adding the side handles makes all the difference. Here is a NeRF of it - trained and rendered in real-time w/ @NVIDIAAIDev Instant-ngp. #3Dscanning #photogrammetry #NeRF #Godox #Smallrig
2
7
119
Through the mirror, again. but this time at an immersive 1:1 scale . NeRF trained and being rendered in real-time on a tethered Quest 2 using @NVIDIAAIDev Instant NeRF. #ai #InstantNeRF #VR #InstantNeRFVR #neuralrendering #computervision
4
23
123
Finally took the plunge into Stable Diffusion & ControNet, using NeRF renders! So much to learn and explore - This might have been a mistake. Apologies. #StableDiffusion #ControNet #NeRF #AUTOMATIC1111
6
16
118
Just rendered this NeRF using @nerfstudioteam's nerfacto - a litter-filled, grimy back alley. Let's acknowledge the less pristine aspects of our urban spaces. Captured with Insta360 RS 1" 360. .#AI #NeRF #nerfacto #neuralrendering #montreal
4
12
119
Unsteady — Montreal's back alleys. [360 video to NeRF to Stable Difussion]. #NERF #StableDiffusion #VFX #AIart
5
21
112
Today's NeRF ft. @nsroy. This synthesis view is captivating - the reflections on the glass doors, the wine glass, the flowers on the table. we even get a sense of the alleyway and the interior of the apartment. The 'floaters' adds the magic. #NeRF #neural #synthesis #volumetric
7
11
110
Our 1st snowstorm of the season just hit and the snow is sticking beautifully on the trees. Perfect start to winter NeRF season at @bonjourdpt ! Trained using @nerfstudioteam + nerfacto. Dataset shot w/ an Insta360 RS 1'' - No🚁. #AI #NeRF #neuralrendering #winter #Montreal
1
9
112
Using magic clay to create layers of shapes and color as a base for revealing landscapes and hidden worlds...I iterated on the magnifying glass real-time tracking to make it better—more stable with some optimizations. Only what is seen through the magnifying glass is fed to
4
13
112
[NeRF]ed self. This view synthesis is simply crazy - 'Instant-NPG'. This is my first run using a photogrammetry data set from on on my volumetric selfies. Thanks @ytcloud_ for the install tutorial and @Azadux for sharing it. #NeRF #neural #synthesis #volumetric
5
13
108
I think this is one of the datasets I've reprocessed most often for testing new features or ideas—219 photos of an alleyway in my neighborhood. #gaussiansplatting #3dgs
5
5
112
Cubifyed @presstube — AR exploration.— 4nd iteration featuring .@bonjourdpt #AR #AugmentedReality #MixedReality #XR #ARKit #ARCore #MadeWithUnity #Unity3D #Montreal
5
21
109
"Disorientation" . → 10/10 on @hicetnunc2000. (In the past year I did a lot of volumetric captures, it became a meditative process). #pointcloud #realtime #3dscan #photogrammetry #realitycapture #madewithunity #CleanNFT #hicetnunc2000
11
17
106
While in NY, I visited the Nintendo store and captured Bowser. Here's a behind-the-scenes and the final output. Real-time render in Unity of a 3D Gaussian Splatting + VFX, iterating on @_kzr 's base project. #GaussianSplatting #RadianceField #Nintendo #nyc #MadeWithUnity
2
11
107
Cubist style transformation — from dining room to fragmented abstraction. #MixedReality #MR #AI #StableDiffusion #SDXLTurbo #Cubism #Quest3
6
11
110
Puddles are fun. jump in! Once again having fun with reflections and NeRF. Trained and rendered using NeRFStudio running nerfacto - @nerfstudioteam...#NeRF #nerfstudio #nerfacto #neuralrendering #mirrorworld #montreal
1
10
104
I tried the XGrids Lixel L2 Pro thanks to Skytech and @XGRIDS2023. This scanner is quite impressive. Here is a capture, rendered as 3D GS, of the staircase of our building I captured while going up the levels. #3dscanning #gaussiansplats #gaussiansplatting #digitaltwin
3
16
108
First rough try at rendering 3D Gaussian Splatting using the @nerfstudioteam mod. I used camera motion from Blender and mixed in some point clouds. There's so much potential. It reminds me of how much I love point clouds. #gaussiansplatting #radiancefield #AI #ML #3D #VFX
4
18
99
Continuing to explore captured scenes and 3D Gaussian splat reconstructions with a triptych monitor setup. #gaussiansplatting #3dgs #madewithunity
3
10
105
The alleyways of Montreal have a unique charm. I have always loved capturing them, especially around my neighborhood. Using a 360 camera in this environment makes things easier and does not significantly impact the enjoyment of walking through them. But using 360 footage is not
3
18
104
I've been training NeRF models using a bunch of my photogrammetry datasets - mostly 'impromptu portraits'. I love both the 'perfect' and 'imperfect' results. I will share some renders over the coming days. Here is one of them. #NeRF #neural #synthesis #volumetric
5
7
101
Latest volumetric portrait. I love the sharpness of the subject, the richness of the space, the contrast of sharpness/focus between the two, the fake sensation of depth in the red capture. @NVIDIAAIDev 's Instant NeRF killing it.. #AI #NeRF #nerfies #neuralnetworks @bonjourdpt
5
29
101
Mixed Reality Diffusion — Captured snapshots from the Quest 3 passthrough feed are processed in real-time with a Stable Diffusion workflow and displayed back on the Quest...#MR #AI #StableDiffusion #ComfyUI #Quest3 #meta #madewithunity
2
21
101
Standing at the shipyard. NeRF ➜ Stable Diffusion ➜ Temporal Kit + EbSynth. . #ai #NeRF #stablediffusion #TemporalKit
6
16
99
My first 'Steganography' self-portrait. I took @alexcarliera's Gaussian Painters for a spin. I'm loving it — My mind's racing with so many ideas. My son had to try it . 😀😶🌫️. #gaussiansplatting #radiancefield #AI #ML #3D #steganography
4
13
99
I made a quick addition to Instant NGP, allowing me to browse a list of NeRFs while in VR. I'm able to teleport between different captures/memories from Montreal, France, Portland. NeRFs trained & rendered w/ @NVIDIAAIDev #InstantNeRF. #AI #VR #InstantNeRFVR #NeuralRendering
3
19
93
Just updated my MR + Diffusion prototype to use @MetaQuestVR Meta’s Passthrough Camera API. It’s a raw port for now; next, I'll focus on UX exploration. It’s now much easier to do a screen recording. #MR #AI #Passthrough @MetaHorizonDevs
7
6
99
Mirror teleportation in @billy_mavreas's lab. I'm getting spine-tingles looking at how the reflection in the mirror has been reconstructed. Synthesized using @NVIDIAAIDev's Instant NeRF.. #AI #NeRF #synthesis #neuralrendering #volumetric #mirror #InstantNeRFSweepstakes
6
17
95
Sometimes, depth just looks better. 3D Gaussian splats of an old dataset from a trashy alleyway, rendered in real-time.
2
8
95
Dreaming of spaceships and space exploration. Video sped up x2. #StableDiffusion #StreamDiffusion #Realtime
4
16
92