One of the great pleasures in life is waking up and immediately going to my computer to check the results of experiments I launched last night
9
27
435
Replies
One of the great pains in life is waking up and finding out the experiments i launched last night had a typo in the launch script
15
12
307
@_jasonwei I haven't felt more broken hearted than the time my 24hr job timed out mere minutes before it could copy the checkpoint from tmp to my home dir. Lessons learnt though.
0
0
0
@_jasonwei I patched this by starting a run 1 hour before bed and sending a Discord notification if it fails.
0
0
0
@_jasonwei Logging off on a Friday only to realise that the training got interrupted on Monday 😩😩
0
0
0
@_jasonwei Maybe a helper AI could catch those - there's some stuff for automating use of your computer that might help with this.
0
0
0