Imagine if people literally argues about , how much more fun you give to the world for training English -> Hylian/Sheikah models?.

Now for another #neuralempty idea for Go to your fav #LLM, make it explain and generate Now take Tatoeba most freq sents and make it generate images in Hylian/Sheikah. Then you find that there are many unsolved NLP tasks =).

@emnlpmeeting @rajammanabrolu And finally:. > Now create more types for animals, bugs, items, junk, etc. and generate 50 items per type. Now you have a dataset enough to visualize/cluster and finetune a model, play around for a workshop paper =).

@emnlpmeeting @rajammanabrolu Then do this to create the dataset:. > Now generate a list of sea creatures and theor animal crossing style quotes, return a JSON for each creature that contains {"type": "sea creature", "name": ". ", "catch_quote": . , "explanation": ". "} Give me a JSONL file.

Want a free #nlproc idea for a paper for at @emnlpmeeting organized by @rajammanabrolu et al.?. Go to the page, then do this to your favorite #LLM:

What if pagerank, tfidf and neural search can co-exist and not cross-canabalize traffic?. What if we rebuild the internet as we knew it? .

Imagine decoy agents making decoy agents and microverse of microverses.

Circling back to this. There’s a huge opportunity to replace humans in a simulation world of “agents”. Imagine instead of paying human vendors to “scale your AI”, which annotators are already aided by models, #llm bots/agents in this other virtual world becomes your vendor.

Ah this is in Table 4 😌.

So many food for thought in one paper! Thanks @BZhangGo @orf_bnw for the fun read!!.

One itch to scratch is really back to the 2B-2B model, what if we just Xavier initialize it instead of initializing with pretrained Gemma?.

BTW, Section 4 data setting is a hidden gem. Why “adapt to a max 2 trillion tokens”? Cos it keeps experiments tractable. We want delta comparisons for ablations not best reasoning model in the world. “Gemma 2 pretaining comes with knowledge distillation.” Got it😉.

@BZhangGo are you folks planning to release the code that does the conversion/initialization of the enc-decoder model with the decoder-only weights?.

I wonder, if we do RLHF eventually why did they still do prefixLM / UL2 “adaptation”. And what happens if we put the adapted en-/decoder model and split into half or average the weights an put it back to a decoder-only model?.

It’s unsatisfying that 9-9B loses to 9B decoder-only in Table3 5-shot WMT. Is it because we give up something in the RLHF for translation?. BTW Table 3 should really be sorted by task name, a bit tedious to compare (a) vs (b).

After reading, re-reading and thinking, why do 2B-2B when you could have split the 2B layers into half and do 1B-1B? That way the 2B decoder-only can be more comparable to 1B-1B en-/decoder.

pretty awesome stuff from #neuralempty folks.

Now we get a glimpse of the flash 😉 #llm #nlproc #renaissance.

Any resemblance is purely coincidental. Someday I hope to see this “bayhugs” software in @misorobotics 😁. #huggingmax #bayhugs.

Olmo from is one of the few commendable and truly transparent #llm