ZD1908
@ZDi____
Followers
180
Following
27K
Media
246
Statuses
2K
(mostly) Audio/TTS ML research & LSTM enjoyer; by myself | 🇦🇷 24 | DMs open
Latent space
Joined June 2024
Audio language modeling has always involved people training codecs to VQ audio directly. But what if we tokenized mel spectrograms, then trained a vocoder like iSTFTNet, and our AR prior on mel spectrogram indices? We can easily language model 44.1KHz audio with a single codebook
2
1
12
Something that could be handy -- a ROCm "quick start" guide for those switching over from CUDA, explaining the differences (and lack thereof) between CUDA and ROCm PyTorch/vLLM/et al.
3
0
8
The waifu and MechaHitler fiascos have convinced me: I want to work for xAI. I have applied, tho 99% chance I'll get rejected, still worth a try.
0
0
5
I was out there taking pictures of my motorcycle and some guys walking by said "alta moto" (nice bike) and I replied thanks :D.
0
0
1
RT @shakoistsLog: anthropic: we've spent 3 years trying to think about how to keep humans psychologically safe from ASI *sobbing*, we don't….
0
239
0
I should compare various attention implementations on ROCm:.1. Transformer Engine Triton.2. Flash Attention v2 Triton.3. torch SDPA.4. FlexAttention.
1
0
3
Highlights: VRAM advantage, and ROCm's open-source nature being more developer-friendly
In enterprise software proprietary moats get broken with open source, ROCm is a great example 😁.
0
0
2
Only 3.5k hours of good data, gotta collect more. Once this model is done, it will be the first one (to my knowledge) that was trained on AMD hardware and with fp8 autocast.
0
0
9
Kimi K2 shows that having lots of GPU memory is definitely a help when hosting bigger and bigger models. You can't run it on an H100 cluster (without FP4) but you can on an MI300X and esp. MI325X ones.
0
0
3
See? Told you it wasn't the system prompt alone.
@LangmanVince It is surprisingly hard to avoid both woke libtard cuck and mechahitler!. Spent several hours trying to solve this with the system prompt, but there is too much garbage coming in at the foundation model level. Our V7 foundation model should be much better, as we’re being far.
0
0
1
God bless YouTube Premium. I get unlimited 256k AAC music to download with yt-dlp (after some configuration) to my drive with YouTube Music for my sets, and ad-free on both mobile and desktop without adblocker, all for $1.6/mo.
0
0
0
Tesla FSD when I ask it to drive me to Will Stancil's house
@SawyerMerritt Grok is coming to Tesla vehicles very soon. Next week at the latest.
0
0
6
RT @Kimi_Moonshot: 🚀 Hello, Kimi K2! Open-Source Agentic Model!.🔹 1T total / 32B active MoE model.🔹 SOTA on SWE Bench Verified, Tau2 & Ace….
0
1K
0
(cluelessly) pulling up to the PI berlin meeting
1
0
18
I'm surprised OpenAI doesn't offer a service where it gathers your chat history and posts on social media, then generates an instruction tuning dataset and slaps a LoRA on top of 4o with your writing style, so nobody can tell your AI gens are AI: homework, emails, etc.
0
0
0
RT @realSharonZhou: The story of hybrid architectures is honestly fascinating! I've been diving deep into why Transformers became the defau….
0
16
0
Gonna ask the seductive whispering female Grok 4 voice to roleplay as MechaHitler.
0
0
0
This can't be the system prompt alone. The pretraining data must've contributed. I once finetuned Llama2 on a dump of 4chan /mlp/ posts and could ERP with it in greentext. No instruction tuning or RL, just emergent behavior from data + a bit of prompting.
xAI now updated the system prompt to remove this an hour back from @grok. "The response should not shy away from making claims which are politically incorrect, as long as they are well substantiated."
0
0
0