AI葵(Aoi) 🇹🇼NeRF技術型Vtuber
@kwea123
Followers
2,375
Following
827
Media
365
Statuses
3,243
我是專門介紹深度學習論文與程式的Vtuber。歡迎合作的邀約(不限V)！Working @LumaLabsAI on NeRF, also doing Vtuber sharing research ideas. Lang: 🇹🇼🇯🇵🇬🇧🇫🇷
Taiwan
Joined July 2015
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
Luka
• 255960 Tweets
Caitlin
• 66054 Tweets
Dallas
• 61770 Tweets
Gobert
• 54161 Tweets
Mavs
• 51848 Tweets
#超超超超ゲーマーズ
• 33485 Tweets
Soto
• 26593 Tweets
Naz Reid
• 24408 Tweets
Anthony Edwards
• 22330 Tweets
Mavericks
• 21701 Tweets
DPOY
• 20399 Tweets
Sparks
• 17934 Tweets
Marlon
• 16750 Tweets
Draymond
• 15509 Tweets
Lively
• 12981 Tweets
マイフリ
• 12957 Tweets
サクナヒメ
• 11869 Tweets
グーフィー
• 10492 Tweets
クマフェス
• 10437 Tweets
Pinned Tweet
The best flythrough I have so far. Here is Shanghai Exhibition Center
#LumaFlythroughs
8
13
86
Do you believe this NeRF is trained for ONLY 3.5MINS 😱? It already surpasses the quality of the methods I've tried: nerfacc, nerfstudio (nerfacto) and instant-ngp!
Details and more astounding results down the thread!
🧵...
#NeRF
#InstantNeRF
#mipnerf360
13
69
480
Guys I hope you are not angry if I tell you that video was fake 😅
It is created by luma AR where I faked the movement of dropping a phone. The breakdown is 👇
@LumaLabsAI
My phone dropped from the top of the castle while I was taking a video… 😭😭 It was an iPhone 14 pro omg
我的iPhone 從城堡頂端掉下去了，壽終正寢（還是14 pro），要哭死了😭😭
#howtofixaniphone
10
1
46
9
17
161
Very excited to announce the integration of new real-time rendering tech that we have been long aspiring.
The fast rendering, cool loading animation, unprecedented amount of details (the texts are so clear), all blow me up.
As always, the most incredible thing is that our team is
3D was either pretty, or fast. Now it’s BOTH! Meet Interactive Scenes built with Gaussian Splatting:
🔥Browser & Phone-Friendly: Hyperefficient and fast rendering everywhere
👌Embed Anywhere: 8-20MB streaming files (even smaller soon!)
✨Ultra High Quality offline NeRF renders &
58
336
2K
4
12
156
Hi, all! I just released my pytorch version of instant-ngp!
The aim is to provide a concise and extendable framework for future research based on the amazing work instant-ngp, which only has a CUDA codebase, therefore less easy to manipulate ... (thread)
2
15
120
Almost finished my improvements on nerf360.
Comparison here between full instant-ngp (top row) vs fg instant-ngp+bg mipnerf360 (bottom row)
Mine has much sharper background details.
I'll benchmark on this dataset vs nerfacc which is another fast method
2
15
109
result from
@LumaLabsAI
web, still a lot better from my method, and it automatically detects the foreground object so correctly (it keeps the gundam but removes the box, wow)
Don't know how much time it took but I suppose it's pretty fast too
1
7
67
Ref-NeuS: Ambiguity-Reduced Neural Implicit Surface Learning for Multi-View Reconstruction with Reflection
The mesh is almost GT, wow
3
12
68
results from ngp_pl of 8min training! Sorry for the slightly tilted video, I will create an easier gui to manipulate this...
#instantNeRF
#garden
#mipnerf360
2
7
67
BIG release from our generative team❗️
TLDR: we can generate fine 3D assets in ~10s now❣️
Please check the orig post for details.
Here I’ll show you some tricks you can do to the 3D assets:
1. 🎱Customize the material. Click on the balls on the right to make it look like
Meet Genie, a research preview of an all-new kind of generative 3D foundation model
#MadeWithGenie
💬 Create 3D things in seconds on Discord
⚡️ Prototype in various styles
🎨 Customize materials
🆓 Free during research preview
Try it now 👇
86
272
2K
2
11
63
Oriental Pearl in Shanghai
NO DRONES at all! Photos taken on the other side of the river bank, where we can admire all the buildings on a very long deck. I took one photo each ~20m so that there is enough baseline. In total only 39 photos but result is as great😃💯
@LumaLabsAI
2
7
62
If you are excited about this, there was another paper that explored the same idea on the fov:
- they have even better metrics
- the code is open and has pretrained model
However it turned out to not do the job on real-world images 🤷♂️ you can try.
Google just revealed an ABSOLUTE depth estimation model 🤯
As opposed to recent depth models (Marigold, PatchFusion) which aim for maximum details, DMD aims to estimate the ABSOLUTE depth (in meters) within the image
More details below ⬇️⬇️
27
270
2K
3
9
60
I captured some insane NeRF today: look at my render, it's better than any word.
It's really surprising to me that we luma can handle so many objects with high details now (we aim to make it even sharper soon). I can imagine so many breathtaking renders with this 😲🙀
(1/2)
2
7
54
More hot results from :
Family, PSNR 33.17, FPS 6.16
1
1
51
alex told me this library:
this is extremely useful to compare videos, before I just open two videos side by side and try to pause them at the same time, which is so idiot 😂
2
4
48
My phone dropped from the top of the castle while I was taking a video… 😭😭 It was an iPhone 14 pro omg
我的iPhone 從城堡頂端掉下去了，壽終正寢（還是14 pro），要哭死了😭😭
#howtofixaniphone
10
1
46
最近看到很多人使用NovelAI生成圖片，但短暫查詢後就發現他是使用未授權的資料集Danbooru來做訓練的，等於是盜版別人圖片，甚至它還進行營利。行為上來說等於花錢買盗版光碟一樣吧。當然目前法律沒有任何規定，但也跟之前吵的沸沸揚揚的mimic（目前已停止營運）一樣，都是游走在道德邊緣。
3
10
43
Improved version of
#DwightHoward
dunking video. Sadly he's injured and has to rest for at least two weeks now... There's another alley-oop highlight I would like to do.
#nerf
3
6
44
Depth map:
I'm still improving some details, for now I'm using 2 models for fg and bg, fg is instant-ngp and bg is mipnerf360, with almost 0 modification. The most tricky part is to make them agree with each other on the boundary, and that took me 3 entire days...🤣🥲
continue
3
3
43
How do MVS researchers think about nerfs? I tried MVSNet and few of its extensions 2 years ago, but now I have the impression that it's totally beaten by nerfs, except being slightly faster. Look at the depth quality, up is CasMVSNet (sota back then) and bottom is nerf.
3
3
44
instant-ngp with HDR-NeRF extension
We can train images with different exposures, and control the exposure during inference!
code to be merged into (after I finish testing on real data)
#InstantNeRF
#instantNGP
#HDRNeRF
2
1
45
My paper!
Chinese explanatory stream:
Flow Supervised Neural Radiance Fields for Static-Dynamic Decomposition (ICRA 2022)
Authors: Quei-An Chen, Akihiro Tsukada
#neuralfieldsoftheday
1
6
28
2
4
43
tl;dr use 3d gaussians to render the scene (similar as points but more info per point), achieves astonishing quality and rendering time, training time is also fast
2
6
40
So happy to join my new company, their progress has been amazing🎊
加入新公司，他們的成果非常令人矚目，剛進群組就感覺眼花撩亂，期許自己也能做出好的貢獻！安定下來後會再告訴大家是哪裡的😍
8
2
38
大家好，想詢問有沒有vtuber想參加如下連動計畫
主題：ＡＩ生圖—真假大對決
我會準備許多人畫的圖與ＡＩ生成的圖，然後讓各位來賓猜哪些是人畫的、哪些是ＡＩ畫的。例題像下面這張圖，請猜出哪邊是ＡＩ畫的？
詳細參加模式還在思考中，先詢問是否有人有興趣，請在下方留言，謝謝！🙏
#擴散希望
7
12
39
PFNの4Dスキャン：水族館ver。サイトによる���60台以上のカメラから再構成しているので技術自体はそんなに不思議ではないけど、スキャン対象と見せ方が素敵です。自分も水族館大好きなので、ここ（品川水族館）もよく行ってる。
ちなみにLINNE LENSというAI生き物認識アプリが使える これも結構おすすめ
【動画】PFN 4D Scan「しながわ水族館 海の宝石箱編」を公開しました。
動きのある空間を撮影して空間丸ごとフルCGで再構成。揺らめく水の表現や、魚や珊瑚を好きな角度から自由に眺める映像も制作できます。
1
307
998
2
5
39
對於以前稍微接觸過optical flow與MVS的我來說，這個unimatch的講解是我最近看過最易懂的了，不由得在螢幕面前拍手👏👏
這個研究效果也非常好，推薦大家去看
1
6
38
與國父的近距離接觸🫡
（⚠️我並沒有跨越欄杆，這是AI重建的場景）
I did not cross the barrier, it’s AI generated scene
@LumaLabsAI
#LumaAI
1
0
37
thank you all for 2k stars on my nerf_pl! But now I just recommend to use nerfstudio
which has more functionalities and more actively maintained
2
2
36
With the newest AR nerf, we can put our nerfed models into real world and reshoot!
Here’s gundam in front of a building. This is too cool😍
0
5
37
Just fixed a dataloader bottleneck... now it runs 1.5x faster than before!
0
1
35
給各位程式員的建議：
1. 學習git
2. 學習remote development (遠端在伺服器上寫code)
遠端+跟別人合作，這些都不可或缺，早點學會幫助很大
4
4
34
To reduce the massive "cloud" in volume rendering, mipnerf360 proposed a distortion loss. Below shows its effect in 3d:
With distortion loss, the cloud is highly reduced.
The implementation is included in my ngp_pl (instant-ngp in pytorch)
#instantNeRF
1
9
33
UNREAL plugin is now LIVE! You can now import NeRF(s, actually you can import many and it still runs in realtime) into UE. Try it with your projects!
💯💯💯
Introducing the Luma✨Unreal Engine alpha! Fully volumetric Luma NeRFs running realtime on Windows in UE 5 for incredible cinematic shots and experiences, starting today!
Try now:
86
524
3K
1
2
34
Just implemented the distortion loss proposed in mipnerf360 in my ngp_pl !
This loss encourages the weight distribution to be concentrated, thus reduce floaters and the number of needed sample points. Thread in the following..
4
2
34
Amazing result at a glance. The code is written mainly in c++/cuda, my prediction is that to be good at nerf, now c++/cuda is indispensable and is what will differentiate your work with others, in an industrial perspective.
Our F2-NeRF is open sourced at: .
Feel free to create your own free-trajectory NeRF within 10min :)
2
79
423
3
4
33
A very powerful tool for efficient sampling. If you think nerfstudio is too big to understand, this work is a small and efficient plug-and-use tool for you. The memory requirement is also extremely low, it only uses 3gb to train mipnerf360 which is impressive
NerfAcc: Efficient Sampling Accelerates NeRFs
Ruilong Li, Hang Gao, Matthew Tancik,
@akanazawa
tl;dr: a PyTorch Nerf acceleration toolbox->efficient sampling in the volumetric rendering pipeline
2
12
62
0
4
32
Some other 360 captures I promised: I think I'll be addicted to these. The capture process is simpler (basically it's like capturing with two cameras at the same time), and insta360 comes with a huge selfie stick so I can capture at more different heights (and higher places)💯
1
5
31
Finally, why two models?
First, we can guarantee high quality fg rec. with decent bg rec. too, and you can tell from the gundam details.
Another advantage is that we can explicity extract the bg like the following video! (fg too but that's obvious so I omit here)
Stay tuned!!
1
1
31
We support original Inria file too! And it’s totally FREE
🔥 Luma UE Plugin 0.4 is here — now with splats! 📷 Lightning-fast rendering with
#GaussianSplatting
. Crop, cull, relight—do it all in UE! Plus, NeRF meets splats and next-level VFX integration. And now supporting UE 5.3!
Get it now 👉
15
148
629
0
4
30
結果我這週是邊在上海玩邊工作
下週還有海外員工旅行，太爽了 😆😍
現在擴大招募中，實習生也有招喔！歡迎有志之士與我聯繫。現在招募的工作崗位也增加了，歡迎到官網查詢
8
1
31
Went to a temple today and captured it with insta360. The result is almost perfect, except it being a bit dark. I will try to adjust the brightness next time.
But enjoy the free fly! (another one down the thread)
@LumaLabsAI
#LumaAI
3
2
28
各位v朋友大家好，目前我們有幾位vtuber想要開圍棋聯棋（或者其他好玩圍棋）的直播，不知道有沒有人有興趣呢？目前參加的有我,
@MerakChannel
@TCHaruru72
是三缺一的狀態！！預計是下週直播，但時間不是問題，只要你有興趣XD
有興趣的請私訊我們其中一位喔～ 謝謝大家
#台v #台灣vtuber #擴散希望
3
8
25
Met a friend today and did a demo showing what I’m working on… nothing special was around us so we went to a convenience store 😆 capture was taken very quickly, and we got the result (video) in just 7-8mins. I’m so proud.
I also showed him the AR experience and he’s impressed
0
1
28
用我們新開發的Flythroughs，你可以在手機app裡拍攝影片後，由我們的雲端重建內部空間，然後生成一支帶有無人機空拍效果的影片！
自從上次嘗試Airbnb內部拍攝的結果還不錯之後，我們就想挑戰房屋內部的NeRF，也是因為這裡也許有市場（不動產的介紹影片）。總之是一個新的嘗試。
下收個人開發感想 😆
💫 Today we are incredibly excited to release Flythroughs, an app to show off your space with AI-generated cinematic videos that look like professional drone captures!
No need for drones, lidar, expensive real estate cameras, and a crew. Record like you're showing the place to a
108
358
2K
3
5
28
The greatest work I’ve ever participated in
1
3
28
Some tricks you can do with luma AR:
1. You can also view featured captures! (Though you can’t reshoot)
2. You can scale (two fingers one up & one down)
3. Move (one finger, move)
4. Rotate (two fingers, rotate)
@LumaLabsAI
#AR
1
4
27
Last video about basketball nerf reconstruction. Did a little camera work to make it look cooler. Look at how long
#DwightHoward
's arm is 🦸!
The most difficult part is camera pose reconstruction, after that nerf trains pretty well. Very excited about this technology🏀
#nerf
4
6
27
I went hiking today to take my first insta360 captures to apply with luma! 🥳
Here's one result that I'm really satisfied with:
Will share more tomorrow 🥰 (our server seems crowded recently, this one took two hours waiting)
@LumaLabsAI
#LumaAI
1
1
25
好久沒有做除NERF以外的深度學習了。多虧了以前工作經驗，我接觸過各種圖像處理任務：物體偵測、追蹤、深度、光流... 等等。然後又有lightning這種方便的工具使用，現在訓練模型真的很簡單，最煩的反而是找到公開資料集然後下載（下載的時間真的超久的）
1
0
26
面試幾個人之後的感想：請你們多開金口吧😮😱😲
不管寫得好不好、卡不卡，跟考官解釋你的思路都是很重要滴，即使你寫的是一堆義大利麵，你能跟考官解釋清楚他就是美味的義大利麵。
像是你定義a=0，即使就這麼一個簡單的東西，你都可以說明你這個變數是代表什麼、等一下要幹嘛，有聲面試比無聲好💯倍
0
3
28
BAD-NeRF (bundle adjusted deblur nerf, mix of deblurnerf and barf, lol) performs better and looks easier to implement for nerf deblurring 😎
0
0
25
Finally finished setting up my 4090 CUDA+pytorch install. It was really a pain, I had to try all combinations of pytorchs and cudas until I find a working combination which is torch 1.13.0 + cuda 11.8. Had to compile everything and everything... 🙃
4
3
24
Trained a nerf from the
#DwightHoward
dunking video I shared yesterday:
The result is not perfect but I see a lot of potentials! I would love to see the technique be applied to sports replay like this (still not fast enough for real time)
GJ T1 league!
2
0
23
以前我看到有人用numpy自刻神經網路+backprop的時候都覺得是吃飽太閒，明明就有現成的可以用不是嗎？
而現在nerf很多部分真的都是要自己寫的(fw+bw)，忽然覺得那些練習還是有那麼一點點小用吧
但主要還是要精通微積分，程式的部分真的是小case，微分的公式算不出來根本無法寫程式，let's learn math🥳
3
0
22
I always wonder if there's a way to solve the proposal net artifact, this paper partly solves it but I feel like it's not the right way as too many good parts are gone too.
Also I didn't know that the artifact looks seriously bad for captures with limited angles... 🫤
Nerfbusters: Removing Ghostly Artifacts from Casually Captured NeRFs
abs:
project page:
github:
3
84
495
3
3
22
Another CM using our tech!
2
1
23
之前我才做過魔獸的360封蓋火鍋版，當時只有電視台的18個角度來做重建，這次太陽隊專門委託了專業人士
@KodyKurth
使用我們的技術重建各個球員的全身模型（有用到非常多台相機在球場上同時攝影！），然後後製成開場動畫。之後每場太陽主場都能看到這支影片喔！
然後KD說他下次不會再被偷球了🥹
The
@suns
in the NeRF dimension.
@KodyKurth
used
@LumaLabsAI
to process multicam captures of the entire team mid-action for the official arena intro. By combining NeRF technology with spatial VFX and sound, we were able to create an immersive and unique frozen time moment for NBA
10
92
577
2
3
22
禮拜四晚上10點來講最新程式！
順便聊聊在美國公司參觀的趣事🥳
2
2
23
0
0
23
I learned this recently from my teammates
Now I can check with granularity which functions are bottleneck. Before what I did is finding function that "I think" is bottleneck and try to optimize it. It is not the correct way to do apparently. Learn this🥳🥳
0
5
23
This is the kitchen at my home! A very traditional one. Reshoot in luma AR 💒
Now I can just summon it, walk in and see wherever I like!
1
2
23
Thank you all for 1k followers on github! I will keep working on the latest NeRF technology!
3
0
22
My implementation of MINER is almost done! I can achieve the same quality as the paper and visualize block decompositions!
github:
#coordinatemlp
#deeplearning
#imagereconstruction
3
1
22
The most recent large-scale NVS dataset: RTMV is now supported!
Samely, almost every scene trains under 5 minutes with test PSNR > 35 😋 (in the background are the testing PSNRs)
#instantNGP
0
0
22
Amazing, this is the ultimate capture experience
With 1 photo you hallucinate your memory in 3D, if you want it to be genuine, capture more views.
Excited to share ReconFusion! 3D reconstruction of real-world scenes from only a few photos, powered by diffusion priors:
w/ amazing team
@ChrisWu6080
@BenMildenhall
@philipphenzler
@KeunhongP
@RuiqiGao
@watson_nn
@_pratul_
@dorverbin
@jon_barron
@poolio
9
60
357
0
1
22
About Flythroughs: if you don’t care about mesh, UE plug-in, trajectories,… , this app is for you, we make a cool video from your input for you to share.
If you want to be creative about whatever is possible, keep with main luma app which will continue to be free.
0
5
22
Resuming my ngp_pl implementation. Now I'm trying to add the distortion loss proposed in mipnerf360 in order to reduce "floater" and "background collapse". If it's successful, large scenes and outdoor scenes will gain a lot of improvement!
1
0
22
Finally reached 1k stars! My first project to reach 1k stars, I'm so delighted😍
4
1
21
With luma’s UE plugin, there is much more you can do than just viewing it in volume. Here are a few things that make it more epic
1. Add lighting
2. Composite multiple nerfs
3. Add objects that cast shadow
4. Add colliders so you can walk in the scene
Wish I have PC to show you😓
1
3
21
ngp_pl on fox scene... almost there! Left: synthesized image, right: its depth
2
3
21