Explore tweets tagged as #WebDatasets
Alright Internet, help me figure out this puzzle with the elusive webdatasets. Why is my .txt file not showing up as a key?!
19
0
13
Get access to #travel, #ecommerce, #job related #webdatasets and much more at 30% discount only on #DataStock. Coupon Code: DS30 Login/ Signup here: https://t.co/U1sx8HWb12 Hurry Up! Offer expires in two days. #data #internet #webdata #sale #offers RT
0
0
1
Popular datasets from major organizations and across the categories like Social Media, eCommerce, Real-estate among others You can also request sample datasets relevant to your use case Try now by signing up using the link below👇 🔗 https://t.co/emRyWVWL6g
1
2
8
Transform data into a valuable asset with our #WebScrapingServices. #Extract, #Clean, analyze #Data for Smarter #Business insights growth. https://t.co/FgWtooepwU
#ActowizSolution #USA #UK #UAE #WebScrapingServicesData #WebScraper #WebDatasets #WebDataColleaction #ScrapeWebData
0
0
0
Compare custom #WebScrapers vs professional #Services. Choose the best solution for your #Business. #Book a demo for expert #WebScraping today! https://t.co/wv79uilDgI
#WebScraper #WebScraping #WebDataExtraction #WebDataExtractor #WebDatasets #WebDataCollection #DataScraping
0
0
0
After much sweat and stress I'm ready to accept that webdatasets is far from optimal for working with audio data.
1
0
1
I will fund someone $25K to make webdatasets significantly better from docs to adding new features. Esp attacking things that drive devs crazy. You'd have to work on it for 20 hours/week. We will give you a list of things and you can make wds better for everyone in AI. DM.
8
10
109
Still not clear to me on all of the magic under the hood, and how it relates to data sharding with libraries like webdatasets, in particular for datasets larger than local memory. Some additional guidance: https://t.co/6wVoa8jkgX
https://t.co/ryWrgvnCB3
1
0
0
If this does what I think it does then this solves one of the most annoying problems when using WebDatasets
Yay! @huggingface datasets==2.20.0 added IterableDataset checkpointing support via torchdata.stateful_dataloader.StatefulDataLoader So instead of figuring out how to rewind the DL on resume, it can now be restored from a checkpoint! This is a super-useful feature: Doc:
2
0
13
Has anyone played around with AIStore ore Webdatasets for PyTorch? I’m really tempted to convert some outdated servers to be AIStore nodes for my lab at Texas State. https://t.co/V7DvKODaRN
https://t.co/z1B4WGXHpo
1
0
0
@vikhyatk I dunno, webdatasets, FFCV, Streaming from Mosaic, all of them were just too quirky to get right with DDP.
1
0
1
There are plenty of cool WebDatasets on HF already: Imagenet-1K https://t.co/TGMAR80rX0 CC12M https://t.co/PgHtHLh3yq
1
0
5
@SanhEstPasMoi @huggingface At that resolution your images start getting a lot of space so it can be more difficult and effective to handle. You can solve the difficulty part with webdatasets or https://t.co/Y9RdbGwpzW but your storage/egress cost is roughly proportional to number of pixels...
1
0
1
@a_e_roberts Cool but does it work for big datasets eg ones stored with webdatasets ?
1
0
5
@karankjariwala @abhi_venigalla @MosaicML Thanks, I’ll try it out. Anyone tried it with Pytorch Lightning distributed training? Using Webdatasets it was kind of a headache
1
0
0
@crisbodnar Why did they kill it? Something like webdatasets I thought doesn't fully address
1
0
0
@girkosh You could easily restructure them as webdatasets 🫣
0
0
1