Daniel Duckworth
@duck
Followers
2,495
Following
143
Media
26
Statuses
330
Researcher/Engineer at Google DeepMind, Berlin.
Berlin, DE
Joined September 2010
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
#بدر_بن_عبدالمحسن
• 323723 Tweets
Meet the Grahams
• 316788 Tweets
#Maythe4thBeWithYou
• 105444 Tweets
Raya
• 89534 Tweets
Saka
• 69029 Tweets
Bournemouth
• 61903 Tweets
어린이날
• 50417 Tweets
Ipswich
• 48768 Tweets
Win in Music Festival
• 47810 Tweets
May the Fourth
• 41806 Tweets
こどもの日
• 41111 Tweets
Leeds
• 40660 Tweets
Havertz
• 39476 Tweets
#ARSBOU
• 37145 Tweets
Sadiq Khan
• 36096 Tweets
Susan Hall
• 30486 Tweets
Declan Rice
• 28452 Tweets
#SixTONESANN
• 27912 Tweets
Birmingham
• 21949 Tweets
Trossard
• 20123 Tweets
サブスク解禁
• 19605 Tweets
ケンタッキーダービー
• 15944 Tweets
右京之介
• 12044 Tweets
Kieran McKenna
• 11345 Tweets
Kierwiński
• 11089 Tweets
Martinelli
• 10438 Tweets
Londoners
• 10027 Tweets
Pinned Tweet
Introducing SMERF: a streamable, memory-efficient method for real-time exploration of large, multi-room scenes on everyday devices. Our method brings the realism of Zip-NeRF to your phone or laptop!
Project page:
ArXiv:
(1/n)
19
191
877
Our paper, “NeRF in the Wild”, is out! NeRF-W is a method for reconstructing 3D scenes from internet photography. We apply it to the kinds of photos you might take on vacation: tourists, poor lighting, filters, and all. (1/n)
76
1K
6K
For lighting and image post-processing, we introduce a low-dimensional embedding space controlling NeRF’s radiance field. This not only gives NeRF-W the capacity to model photo-specific lighting, it enables us to “relight” a scene from new angles. (3/n)
5
33
227
This project wouldn’t have been possible without my amazing coauthors:
@rmbrualla
, Noha Radwan, Mehdi S. M. Sajjadi,
@jon_barron
, and Alexey Dosovitskiy. Check out our paper:
6
7
98
We build on NeRF, a method for learning a volumetric radiance field from a posed photo collection. We introduce two extensions to soften NeRF’s “static world” assumption: one for lighting/post-processing, the other for transient objects. (2/n)
1
8
84
NeRF-W improves on the SOTA by >5dB in PSNR and reduces error on other metrics by 20-50%. Qualitatively, NeRF-W produces consistent, crisp 3D geometry without fog or checkerboard artifacts. Check out the project website for more videos and the paper. (5/n)
2
8
82
For transient objects, we introduce a secondary volumetric radiance field combined with an uncertainty field. The former explicitly captures transient objects; the latter uncertainty about the color of a pixel passing through part of the 3D space. (4/n)
3
5
75
SMERF has the best of both worlds: we produce renders nearly indistinguishable from Zip-NeRF while rendering at 60 fps or more on desktops, laptops, and even recent smartphones, all while scaling to scenes as big as a house!
(3/n)
3
6
53
How does one trade-off sample quality and diversity in a language model? Which decoding method is best? We introduce a multi-objective framework maximizing human judgement score subject to a constraint on diversity (entropy). (1/7)
2
8
39
@Knusper2000
We used a few hundred to low-digit thousands. Based on the results of NeRF, if you capture your images in a controlled environment, you might be able to get away with as few as one hundred!
3
4
36
@NextWorldOfTech
Infinite! We don't use polygons. We learn a "volumetric radiance field" scene representation.
4
0
34
This has been a joint work with my amazing collaborators:
@PeterHedman3
,
@chrisjreiser
,
@PeterZhizhin
,
@jfthibert
,
@mariolucic_
,
@RSzeliski
, and
@jon_barron
Learn more and try SMERF out yourself at
(8/n)
2
4
26
How do we achieve this? We distill a teacher model into a family of MERF-like student submodels, each of which specializes to a different part of the scene. Each submodel captures the entire scene, so rendering stays fast and GPU memory consumption stays low.
(4/n)
1
0
23
@pradeepviswav
@benedictevans
@photosynth
@MSFTResearch
Indeed, we were heavily inspired by Photosynth! I remember being in awe when I first saw it back in high school.
2
0
20
Only a single submodel needs to be in memory at a time, and while the user explores the space, we swap out old submodels and stream in new ones. We train submodels to be mutually consistent, making transitions barely noticeable.
(6/n)
1
0
21
We also modify MERF to significantly improve visual fidelity on small-to-medium size scenes. Our submodels capture thin geometry, high-resolution textures, and specular highlights better than ever before.
(5/n)
1
1
18
The result: a set of compact, streaming-ready submodels ready to run at up to 60 fps in your browser. The best part: you can try it out yourself:
(7/n)
1
1
16
@zellyn
@pradeepviswav
@benedictevans
@photosynth
@MSFTResearch
Like Photosynth, we require one to run a registration pipeline such as COLMAP to derive camera parameters (position, direction, focal length, etc). Once that's done, we learn the scene representation.
0
2
16
Existing approaches for view-synthesis are torn between two conflicting goals: high quality and fast rendering. Most methods only achieve one or the other.
(2/n)
1
1
15
For all -1 of you reading this, I started a blog @
http://t.co/U9M3bZFAJx.
It's almost entirely optimization proofs. LAGRANGIANS TO THE FACE
0
4
9
Proud to have my first arXiv publication with Sam, Jascha, and Quoc!
Stochastic natural gradient descent corresponds to Bayesian training of neural networks, with a modified prior. This equivalence holds *even away from local minima*. Very proud of this work with Sam Smith, Daniel Duckworth, and Quoc Le.
2
36
125
0
0
6
I'm stoked to be a contributor on Object SRT, a new method for unsupervised, posed-images-to-3D-scene representation and segmentation! It's crazy fast and, while far from perfect, is leaps and bounds better than anything I've seen yet :)
So excited to share Object Scene Representation Transformer (OSRT):
OSRT learns about complex 3D scenes & decomposes them into objects w/o supervision, while rendering novel views up to 3000x faster than prior methods!
🖥️
📜
1/7
5
119
647
0
0
7
Chen et al's "Neural Ordinary Differential Equations" () is named one of NeurIPS's best papers and featured on
@techreview
. Cool, novel work!
2
0
6
I'm proud to announce the release of our new paper relating Whitening, Newton's Method, and Generalization! tl;dr whitening w/o regularization significantly reduces a model's ability to generalize.
Work with
@negative_result
@sschoenholz
@ethansdy
@jaschasd
Whitening and second order optimization both destroy information about the dataset, and can make generalization impossible: We examine what information is usable for training neural networks, and how second order methods destroy exactly that information.
9
113
551
3
0
6
@RadianceFields
Want to learn more? Check out my OG explainer thread!
Introducing SMERF: a streamable, memory-efficient method for real-time exploration of large, multi-room scenes on everyday devices. Our method brings the realism of Zip-NeRF to your phone or laptop!
Project page:
ArXiv:
(1/n)
19
191
877
0
1
5
Super proud of my ACL publication with
@daphneipp
! tl;dr we find that the decoding methods that produce the most "human-like" text are also the easiest for BERT-style classifiers to identify. We humans and our models don't see text the same way!
1
0
5
Command line parameters overriding config file contents? Yeah, configurati does that.
0
0
5
@zhenjun_zhao
@PeterHedman3
@ChrisJReiser
@PeterZhizhin
@MarioLucic_
@jon_barron
Don't forget our amazing coauthors,
@jfthibert
and
@RSzeliski
!
0
0
3
None of this would be possible without my amazing collaborators!
@negative_result
,
@sschoenholz
,
@ethansdyer
, and
@jaschasd
.
0
0
3
But all is not lost! We also find that *regularized* second-order optimization leads to better generalization than un-regularized second-order optimization or gradient descent.
2
0
3
@w4nderlus7
I believe and this work aren't in conflict! Two points: (1) The Meena paper says, "given two models, the one with better perplexity produces better samples." This work says, "given two samples, the more likely isn't always better."
2
0
3
@Samu_tweetz
I'm living in SF, but I'll be in Europe till October. Let's have a reunion when I get back!
0
0
2
@NextWorldOfTech
It is, but the scene representation is a "volumetric radiance field". I really like the original presentation by the NeRF authors on the subject:
1
1
3
@Kyrannio
Don't forget my amazing collaborators at Google Research, Google Inc, and Tübingen! This was very much a team effort.
More info here:
Introducing SMERF: a streamable, memory-efficient method for real-time exploration of large, multi-room scenes on everyday devices. Our method brings the realism of Zip-NeRF to your phone or laptop!
Project page:
ArXiv:
(1/n)
19
191
877
2
1
2
Have you wondered how effective social distancing is? Or quarantining? What happens if a few people ignore social distancing? How bad is it to go to the grocery store?
In short, everything helps -- especially early testing and quarantine! We're all in this together.
New video: Simulating an epidemic.
What happens when people avoid each other for the most part but still go to a common central location like a store?
What if you can track and isolate cases, but 20% slip through the cracks? 50%?
And much more.
111
2K
6K
1
0
1
Key takeaways: (i) very high likelihood samples are bad, (ii) compare decoding methods fairly by controlling entropy, and (iii) there's more to decoding methods than favoring high-likelihood samples. (6/7)
1
0
2
A new day, a new proof. That's right kiddies, Accelerated Proximal Gradient Descent works.
http://t.co/mzvyBVKA0f
0
0
2
Spot on article on the state of AI and the Mind. Definitely worth the read!
"Despite the remarkable commercial success of current AI systems...we still have a long way to go in mimicking truly human like intelligence."
1
0
2
When using log likelihood as a proxy for human judgement ("quality"), we obtain "Global Temperature Sampling", a globally-normalized decoding method that optimally traverses the quality-diversity curve. (2/7)
1
0
2
@conroydave
Thanks for the heads up. It looks like it's running to me, except for a few dropped images. Will fix ASAP.
2
0
2
@johnmyleswhite
I'm happy to switch as long as I'm not fighting (too many) compiler bugs! I'll give it a shot, it can't hurt.
1
0
2
While this blog post may only have two authors, the project itself is the hard work of a number of amazing teammates. Take a peak at the "Acknowledgments" section -- you may spot a few familiar names :)
0
0
2
@w4nderlus7
On the decoding side, the Meena paper also advocates for a sample-and-rank method, N=20. I hypothesize that the method doesn't surface decodes on the "too likely" side of the Likelihood Trap. We didn't compare sample-and-rank as...
1
0
2
@weargustin
I missed your Kickstarter! Any way to get an order in? If not, any friends you recommend in your place?
1
0
2
@supremebeme
@RadianceFields
This and other large scenes are captured with a DSLR camera and a fisheye lens. Approximately ~1500 photos are used. Capture takes 30~60 min.
1
0
2
We perform the first large-scale human study (>38,000 ratings) comparing decoding method/hyperparameter combinations against each other. When controlling for entropy, we find nucleus > top-k > temperature sampling in low-entropy regimes. (4/7)
1
0
2
Just published a Kalman Filter library for Python. Go check it out! documentation:
http://t.co/PTEXRe0w
, and source:
2
0
2
@WholeMarsBlog
Want to learn more? Check out my OG explainer thread:
Introducing SMERF: a streamable, memory-efficient method for real-time exploration of large, multi-room scenes on everyday devices. Our method brings the realism of Zip-NeRF to your phone or laptop!
Project page:
ArXiv:
(1/n)
19
191
877
0
0
1
Thanks to
@jdanbrown
, Mirai is cleaner and safer than ever. Mirai: multithreading done right for Python!
http://t.co/Nfq4tkMWcZ
0
0
1
We further find that, when pairing samples from decoding methods with random samples from the model *with equal likelihood*, temperature sampling is preferred to nucleus and top-k sampling by human raters. (5/7)
1
0
1
We fit a small CNN, MLPs, and a linear model w/ and w/o Natural Gradient Descent (a second order optimizer) and find all generalize more poorly than those trained w/ gradient descent.
1
1
1
@kyledoherty
list comprehensions are a higher form of happiness, directly followed by grilled cheese sandwiches.
0
0
1
Surprisingly, this method is *worse* than token-by-token decoding methods according to human raters! We discover this is a consequence of the "Likelihood Trap", wherein samples with exceptionally high likelihood receive low human judgement scores. (3/7)
1
0
1
This is a highly unintuitive result! In linear regression, training on a whitened dataset w/ fewer data points than dimensions results in a model that *cannot* of doing better than random chance on a validation set!
1
0
1
@AISavvyKushan
Not just DeepMind! This wouldn't have been possible without my amazing colleagues in Google Research, Google Inc, and Tübingen.
1
0
1
@ali_thespaceguy
We accomplished this using a couple thousand photos from the Image Matching Challenge 2020 dataset.
0
0
1
@bengeliscious
If your observations can be expressed as integrals a la CT scans, I don't see why not :)
1
0
1
From the moment NeRF was first published, the research community knew it would be something game-changing. I'm proud to be part of the team turning this amazing line of work into a real product experience!
Immersive View gives users a virtual, close-up look at indoor spaces in 3D! Learn how it uses neural radiance fields to seamlessly fuse photos to produce realistic, multidimensional reconstructions of your favorite businesses and public spaces →
16
98
370
2
0
1
Further, applying gradient descent on a whitened dataset is *exactly* equivalent to applying Newton's Method on the original dataset. This suggests that models trained w/ second order methods may generalize as well as those trained w/ SGD.
1
0
1
Goodbye,
@rxbofficial
. It's been a wonderful 5 years to know you, and I look forward to whatever comes next.
0
0
1
@wxswxs
The original NeRF folks produced depth maps and meshes learned from those maps on their project website. You can see our depth map of Trevi Fountain in the overview video @ 2:25
1
0
1
Here’s a head-to-head comparison of nucleus, top-k, and temperature sampling, and our newly proposed decoding method. Sampling directly from the model is by far the worst and nucleus p=0.3 is best according to human judgement. (7/7)
1
0
1