What is the pathway towards greater diversity in data and AI? Hear from Professor Ruth Okediji, scholar of IP Law at Harvard Law School, who will be in conversation with Assistant Dean Amanda Watson of the Harvard Law School Library on Oct 22 at 2PM. https://t.co/20ymDIQmVK

What is the pathway towards greater diversity in data and AI? Hear from Professor Ruth Okediji, scholar of IP Law at Harvard Law School, who will be in conversation with Assistant Dean Amanda Watson of the Harvard Law School Library on Oct 22 at 2PM. https://t.co/20ymDIQmVK

Join us tomorrow at 10AM EST: https://t.co/FakraXOEzv

Register to join the talk virtually:

Can a small visual language model read documents as effectively as models 27 times its size? Next Friday, IDI will host Michele Dolfi and Peter Staar from @IBMResearch Zurich to discuss their work on SmolDocling, an “ultra-compact” model for diverse OCR tasks.

This Monday, @instdin will host @petrknoth to share his experience leading CORE ("The world’s largest collection of open access research papers") as the rise of AI brings new meaning, and challenges, to stewarding knowledge repositories. Join us virtually via the link below.

Tomorrow, it's our pleasure to host @ayahbdeir to talk about the power of data in building an AI ecosystem that's open, transparent, and fair. 11am ET on June 17th. Register at the link below to attend virtually. Cohosted by the @instdin and @BKCHarvard.

We hope Institutional Books will be the beginning of a process that makes millions more books accessible to the public for a variety of uses. We welcome feedback as we continue to expand this dataset, refine its contents, and sharpen our process. https://t.co/gPuXtKAayI

We look forward to growing Institutional Books through community. We welcome collaboration from researchers and model makers as we: - Evaluate the dataset’s impact on model outputs - Continuing to refine our OCR pipelines View the dataset on Hugging Face:

As part of our refinement work, we supplemented the original OCR-extracted text with a post-processed version that utilizes line detection to reassemble the text according to the line type.

We included extensive volume-level metadata with both original and generated components, such as results from text-level language detection.

We analyzed the dataset’s coverage across time, topic, and language and found: - 40% of English text + long tail of 254 languages - 20 clear topical tranches - Largely published in the 19th and 20th centuries Technical report here: https://t.co/Z7gvo4qCoe

Today we released Institutional Books 1.0, a 242B token dataset from Harvard Library's collections, refined for accuracy and usability. 🧵

I've loved writing words, while loops and wandering wectors, so I'm thrilled to join the @instdin team at Harvard as the director of community and communications! https://t.co/B6ZgWRAevG

As the Institutional Data Initiative (@instdin) expands its mission, we’re announcing a collaboration with the Boston Public Library (@BPLBoston) to develop AI-driven tools capable of accelerating new digitization at libraries across the world, starting at BPL. 🧵

I'm pleased to announce we're expanding our mission at the Institutional Data Initiative (@instdin) with an open call for institutional collaborators, new digitization at Harvard Law School Library, and additional support to advance this work.

Today we're launching the Institutional Data Initiative to work with libraries, gov agencies, and other knowledge institutions to help refine and publish their collections as data, with an eye toward AI. 🧵

https://t.co/mOPZzrbx9I

Hello world. 🧵